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FOREWORD 


Stories  about  unmanned  vehicles  now  appear  reg¬ 
ularly  in  the  national  news  —  and  not  always  in  good 
ways.  Within  the  last  2  years,  privately-owned  drones 
have  crashed  on  the  White  House  lawn,  surveilled  the 
U.S.  Embassy  in  Paris,  and  buzzed  German  Chancel¬ 
lor  Angela  Merkel.  When  used  in  military  operations, 
autonomous  weapon  systems  (AWS)  have  the  poten¬ 
tial  to  save  lives  as  well  as  apply  lethal  force  across 
land,  sea,  and  air. 

In  this  Letort  Paper,  Mr.  Jeffrey  Caton  posits  that 
the  development  of  AWS  policy  and  doctrine  should 
characterize  autonomy  not  as  a  discrete  property  of 
a  given  system,  but  rather  as  a  function  that  varies  in 
its  strategic,  operational,  and  tactical  context  and  mis¬ 
sion  application.  Further,  he  argues  that  AWS  design, 
planning,  and  operations  should  be  tempered  with 
the  purposeful  consideration  of  human  judgment 
and  control  as  well  as  legal  and  ethical  standards  that 
foster  international  credibility. 

Through  its  current  military  operations,  the  United 
States  is  setting  both  overt  and  tacit  precedents  for  the 
world  with  regard  to  the  appropriate  use  of  AWS.  This 
Paper  provides  readers  with  background  information 
crucial  to  the  full  understanding  of  the  complex  chal¬ 
lenges  facing  the  future  development  and  operation  of 
AWS  across  the  full  range  of  military  operations. 


DOUGLAS  C.  LOVELACE,  JR. 
Director 

Strategic  Studies  Institute  and 
U.S.  Army  War  College  Press 
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SUMMARY 


The  use  of  autonomous  weapon  systems  (AWS)  in 
warfare  is  not  a  new  concept.  One  could  argue  that  the 
development  and  integration  of  such  systems  have 
evolved  over  the  last  century  and  accelerated  signifi¬ 
cantly  after  the  September  11,  2001  (9/11)  attacks  on 
the  U.S.  homeland.  AWS  will  likely  continue  to  grow 
in  both  capabilities  and  numbers.  This  growth  of  un¬ 
manned  systems  is  not  limited  to  the  U.S.  military  or 
even  the  United  States;  it  is  an  international  phenom¬ 
ena  that  includes  government  and  commercial  appli¬ 
cations  in  all  domains  — air,  land,  and  sea.  Commercial 
endeavors  for  unmanned  systems  are  at  the  forefront 
of  many  technologies  and  their  proliferation  will 
likely  outnumber  military  use  in  the  future. 

What  does  the  Department  of  Defense  hope  to  gain 
from  the  use  of  AWS?  This  Letort  Paper  explores  a  di¬ 
verse  set  of  complex  issues  related  to  the  developmen¬ 
tal,  operational,  legal,  and  ethical  aspects  of  AWS.  It 
will  briefly  explore  the  recent  history  of  the  develop¬ 
ment  and  integration  of  autonomous  and  semi-auton- 
omous  systems  into  traditional  military  operations.  It 
will  examine  anticipated  expansion  of  these  roles  in 
the  near  future  as  well  as  outline  international  efforts 
to  provide  a  context  for  U.S.  use  of  the  systems.  These 
topics  are  well-documented  in  many  sources,  thus  this 
Paper  serves  as  a  primer  for  current  and  future  AWS 
operations  to  provide  senior  policymakers,  decision¬ 
makers,  military  leaders,  and  their  respective  staffs  an 
overall  appreciation  for  existing  capabilities  and  the 
challenges,  opportunities,  and  risks  associated  with 
AWS  across  the  range  of  military  operations.  Empha¬ 
sis  is  added  to  missions  and  systems  that  include  the 
use  of  deadly  force. 
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Discussion  is  limited  to  unclassified  and  open 
source  information;  any  classified  discussion  must  oc¬ 
cur  at  another  venue.  Despite  the  goal  of  using  precise 
language,  the  following  terms  may  be  used  somewhat 
interchangeably  due  largely  to  the  variety  of  source 
material  cited  in  this  Paper:  autonomous  weapon  sys¬ 
tem;  robot;  drone;  unmanned  system;  unmanned  ve¬ 
hicle;  unmanned  platform. 

After  providing  background  information  on  the 
meaning  of  autonomy  and  its  contemporary  applica¬ 
tions,  the  Paper  establishes  a  dialogue  in  four  main 
sections: 

1.  Developmental  Issues  Related  to  Autonomous 
Systems.  This  section  explores  two  questions:  How 
should  the  United  States  develop  autonomy-related 
applications  for  use  across  the  range  of  military  op¬ 
erations?  What  technologies  are  imperative  to  the  de¬ 
velopment  of  AWS?  It  addresses  these  questions  first 
by  describing  the  current  state-of-the-art  for  Army 
unmanned  ground  vehicle  systems,  then  by  examin¬ 
ing  technology  issues  across  the  scope  of  application 
of  autonomy,  and  ends  with  a  brief  analysis  of  the  fed¬ 
eral  budget  trends  for  AWS  development. 

2.  Operational  Issues  Related  to  Use  of  Autono¬ 
mous  Systems.  This  section  addresses  two  questions 
related  to  a  vision  of  fully  integrated  AWS  operations: 
What  can  be  achieved  via  autonomy  within  the  range 
of  military  operations?  What  missions  can  be  en¬ 
hanced  by  incorporating  AWS?  The  resulting  discus¬ 
sion  considers  the  role  of  mission  parameters,  efforts 
toward  AWS  integration  into  force  structure,  doctri¬ 
nal  requirements,  international  efforts,  and  strategic 
geopolitical  challenges. 

3.  Legal  Issues  Related  to  Use  of  Autonomous 
Systems.  This  section  examines  the  legally  acceptable 
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uses  of  autonomous  weapon  systems  within  the  range 
of  military  operations.  It  first  looks  at  existing  interna¬ 
tional  legal  conventions  including  the  Law  of  Armed 
Conflict,  then  reviews  recent  assessments  by  the  Unit¬ 
ed  Nations,  next  it  focuses  on  the  concept  of  human 
control  and  judgement  as  it  applies  to  AWS.  Finally,  it 
reviews  current  vignettes  as  well  as  consider  potential 
trends  for  the  future. 

4.  Ethical  Issues  Related  to  Use  of  Autonomous 
Systems.  What  should  be  the  ethically  acceptable  and 
advisable  uses  of  AWS  within  the  range  of  military 
operations?  What  moral  principles  should  form  the 
foundation  of  AWS  development  and  operation?  This 
section  explores  these  questions  by  first  reviewing  the 
ongoing  work  for  developing  ethical  frameworks  for 
AWS,  then  considering  the  varying  cultural  views  that 
AWS  applications  may  evoke,  as  well  as  analyze  the 
potential  reduction  or  proliferation  in  warfare  that 
widespread  use  of  AWS  may  introduce.  Finally,  it 
looks  toward  the  future  and  contemplates  potential 
long-term  effects  on  national  security. 
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AUTONOMOUS  WEAPON  SYSTEMS: 

A  BRIEF  SURVEY  OF  DEVELOPMENTAL, 
OPERATIONAL,  LEGAL,  AND  ETHICAL  ISSUES 


The  use  of  autonomous  weapon  systems  (AWS)  in 
warfare  is  not  a  new  concept.  One  could  argue  that  the 
development  and  integration  of  such  systems  have 
evolved  over  the  last  century  and  accelerated  signifi¬ 
cantly  after  the  September  11,  2001  (9/11)  attacks  on 
the  U.S.  homeland.  AWS  will  likely  continue  to  grow 
in  both  capabilities  and  numbers.  This  growth  of  un¬ 
manned  systems  is  not  limited  to  the  U.S.  military  or 
even  the  United  States;  it  is  an  international  phenom¬ 
ena  that  includes  government  and  commercial  appli¬ 
cations  in  all  domains  —  air,  land,  and  sea.  Commercial 
endeavors  for  unmanned  systems  are  at  the  forefront 
of  many  technologies  and  their  proliferation  will 
likely  outnumber  military  use  in  the  future. 

BACKGROUND 

Before  examining  issues  related  to  autonomous 
weapon  systems,  it  is  important  to  establish  a  broad 
context  and  lexicon  for  the  subject.  This  section  ex¬ 
plores  the  meaning  of  autonomy  and  establishes  the 
definition  for  use  in  this  Paper.  It  also  provides  a 
context  for  discussion  through  a  brief  look  at  the  his¬ 
tory  of  autonomous  systems  as  well  as  contemporary 
applications. 

What  is  Autonomy? 

The  intellectual  dialogue  among  philosophers  re¬ 
garding  autonomy  in  human  interactions  goes  back 
over  300  years.1  There  is  no  universally  accepted  defini- 
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tion  for  autonomy  for  applications  involving  human- 
machine  systems.  This  should  not  be  a  surprise,  given 
the  complex  nature  and  implications  of  anonymous 
systems,  and  such  a  lack  of  basic  terminology  is  not 
unique  (e.g.,  consider  the  vastly  different  meanings  of 
the  term  "cyberspace"  in  contemporary  culture).  There 
appears  to  be  a  general  consensus  within  the  ongoing 
discourse  on  the  topic  that  autonomy  is  best  character¬ 
ized  not  as  a  discrete  property  of  an  object  or  system, 
but  rather  as  a  relationship  between  a  system  and  its 
operator  that  may  vary  across  the  spectrum  of  differ¬ 
ent  degrees  of  system  autonomy.  In  simple  terms,  this 
spectrum  progresses  from  controlled  operations  ("hu¬ 
man  in  the  loop"),  to  supervised  operations  ("human 
on  the  loop"),  to  fully  autonomous  operations  ("hu¬ 
man  out  of  the  loop").2  Thus,  the  use  of  machine  au¬ 
tonomy  is  a  common  experience  in  daily  life  when  one 
considers  such  systems  as  automobile  cruise  control, 
aircraft  autopilot,  or  digital  video  recorders. 

When  one  considers  the  use  of  autonomy  in 
weapon  systems,  the  need  for  more  precise  language 
is  required,  especially  for  systems  that  may  involve 
the  potential  application  of  lethal  force.  Toward  this 
end,  the  United  States  is  one  of  the  few  countries  in 
the  world  to  have  an  openly  published  government 
policy  in  this  area.3  Department  of  Defense  (DoD)  Di¬ 
rective  3000.09,  Autonomy  in  Weapon  Systems,  not  only 
establishes  policy  and  organizational  responsibilities, 
but  it  also  "establishes  guidelines  designed  to  mini¬ 
mize  the  probability  and  consequences  of  failures  in 
autonomous  and  semi-autonomous  weapon  systems 
that  could  lead  to  unintended  engagements."4  In  ad¬ 
dition  to  "autonomous"  and  "semi-autonomous,"  the 
directive  also  defines  "human-supervised"  as  a  cat¬ 
egory  of  autonomous  weapon  systems.  Figure  1  pro- 
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vides  the  definitions  of  these  categories  of  autonomy 
with  their  correlated  common  names.  The  definitions 
are  arranged  in  increasing  level  of  autonomy  with 
an  intentional  overlap  between  the  graphic  borders 
around  each  definition.  These  intersecting  areas  pro¬ 
vide  conceptual  clarity  since  a  given  system  may  have 
subsystems  operating  in  different  modes  of  autonomy 
during  different  parts  of  a  mission.  For  simplicity,  this 
Paper  will  use  the  term  "autonomous  weapon  sys¬ 
tem"  or  AWS  to  describe  any  DoD  system  that  fits  the 
definitions  shown  in  this  Figure. 


Semi-autonomous 

A 

Human-supervised 

A 

Fully  Autonomous  | 

"Human  in  the  loop" 

u 

"Human  on  the  loop" 

V 

"Human  out  of  the  loop" 

Weapon  system  that, 
once  activated,  is 
intended  to  only  engage 
individual  targets  or 
specific  target  groups 
that  have  been  selected 
by  a  human  operator. 

Includes  "fire  and  forget" 

\  munitions 

An  autonomous  weapon 
system  that  is  designed 
to  provide  human 
operators  with  the 
ability  to  intervene  and 
terminate  engagements, 
including  in  the  event  of 
a  weapon  system  failure, 
before  unacceptable 
levels  of  damage  occur. 

A  weapon  system  that, 
once  activated,  can  select 
and  engage  targets 
without  further 
intervention  by  a  human 
operator. 

) 

(Note:  Definitions  per  DoDD  3000.09,  Autonomy  in  Weapon  Systems) 


Figure  1:  Spectrum  of  Autonomy  in  Weapon 
Systems.5 
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It  is  important  to  note  that  DoD  clearly  distinguish¬ 
es  a  difference  between  autonomy  and  remote  control. 
Thus,  when  considering  unmanned  aircraft,  their  rule 
is  "when  the  aircraft  is  under  remote  control,  it  is  not 
autonomous.  And  when  it  is  autonomous,  it  is  not 
under  remote  control."6  This  distinction  implies  that 
any  remote  control  of  an  unmanned  aircraft7  negates 
any  consideration  of  autonomy.  However,  in  a  com¬ 
plex  system-of-systems  design,  the  operator  may  be 
remotely  controlling  a  sensor  while  the  aircraft  is  on 
autopilot— essentially,  the  aircraft  is  flying  itself  and 
should  be  considered  autonomous.  The  implications 
for  a  more  nuanced  model  of  autonomy  are  discussed 
later  in  this  Paper. 

While  DoD  Directive  3000.09  applies  to  AWS 
that  involve  "the  application  of  lethal  or  non-le- 
thal,  kinetic  or  non-kinetic,  force  by  autonomous  or 
semi-autonomous  weapon  systems,"  it  specifically 
excludes: 

cyberspace  systems  for  cyberspace  operations;  un¬ 
armed,  unmanned  platforms;  unguided  munitions; 
munitions  manually  guided  by  the  operator  (e.g.,  la¬ 
ser-  or  wire-guided  munitions);  mines;  or  unexploded 
explosive  ordnance.8 

This  distinction  of  applicability  may  introduce  a  dis¬ 
connect  into  DoD  policy  with  respect  to  "unarmed, 
unmanned  platforms"  since  they  may  still  inflict  inju¬ 
ry  or  collateral  damage  to  individuals  and  property  if 
they  malfunction.  For  example,  an  automated  convoy 
vehicle  that  runs  amok  due  to  a  system  malfunction 
may  injure  a  person  or  cause  damage  that  is  similar  in 
effect  to  collateral  damage  from  an  errant  AWS. 

A  vital  theme  in  DoD  Directive  3000.09  is  the  re¬ 
quirement  for  AWS  to  "be  designed  to  allow  com- 
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manders  and  operators  to  exercise  appropriate  levels 
of  human  judgment  over  the  use  of  force."9  Respon¬ 
sibility  to  meet  this  mandate  is  shared  among  many 
facets  of  the  development  and  procurement  process, 
to  include  rigorous  hardware  and  software  testing, 
safety  and  information  assurance  precautions,  op¬ 
erator  procedures  and  training,  and  legal  review.  The 
section  of  this  Paper  on  operational  uses  will  discuss 
issues  related  to  these  obligations  in  more  detail. 

Historical  Context. 

The  use  of  AWS  in  warfare  is  not  a  new  concept. 
One  could  argue  that  the  development  and  integra¬ 
tion  of  such  systems  has  evolved  over  the  last  century. 
While  a  comprehensive  exploration  is  beyond  the 
scope  of  this  work,  some  famous  examples  include  the 
Kettering  "Bug"  Aerial  Torpedo  (circa  1917)10  as  well 
as  Soviet  "Teletank"  units  (two  battalions),  and  Ger¬ 
man  "Goliath"  remote-control  mini-tanks  (over  7,500 
produced)  used  during  World  War  II.11  Evolutionary 
system  milestones  that  may  be  of  significant  interest 
to  modern  AWS  include  the  first  launch  of  a  Maverick 
missile  from  an  Air  Force  target  drone  in  December 
197112  and  the  introduction  in  1972  of  the  British  Mor- 
fax  Wheelbarrow,  the  first  unmanned  ground  vehicle 
(UGV)  used  for  explosive  ordnance  disposal  (EOD).13 

Looking  exclusively  at  UGVs,  Douglas  Gage  of  the 
Naval  Ocean  Systems  Center  has  written  about  tech¬ 
nical  aspects  of  the  early  days  of  these  ground-based 
AWS.14  Gage's  account  characterizes  the  efforts  as 
largely  focused  on  research  and  development  in  the 
1960s  through  the  1980s,  with  work  led  by  the  Defense 
Advanced  Research  Projects  Agency  (DARPA)  and 
prestigious  technical  universities.  Various  programs 
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experimented  with  various  forms  of  locomotion,  sens¬ 
ing,  navigating,  and  decisionmaking  as  well  as  the 
ability  to  manipulate  tools  to  perform  a  specific  mis¬ 
sion.  In  the  1980s,  research  programs  emerged  that 
were  more  focused  on  potential  military  applications. 
Among  them  were  the  U.S.  Army  Tank- Auto  motive 
Command  (TACOM)  and  DARPA  work  on  the  Ad¬ 
vanced  Ground  Vehicle  Technology  and  the  Naval 
Ocean  Systems  Center  work  on  Ground  Surveillance 
Robot  (GSR)  for  the  Marine  Corps.  The  GSR  was  not  a 
small  system;  it  used  the  7-ton  M-114  armored  person¬ 
nel  carrier  as  its  foundation.  Other  efforts  during  this 
time  period  used  dune  buggies  and  the  High  Mobility 
Multipurpose  Wheeled  Vehicles  (HMMWVs)  as  their 
chassis.15  Development  of  robotic  systems  in  the  1980s 
also  included  some  of  the  early  armed  UGVs  such  as 
the  Grumman  Robotic  Ranger  (demonstrated  remote 
missile  firing)  and  the  RDS  PROWLER  (demonstrated 
missile  and  machine  gun  firing)  as  well  as  several 
prototypes  of  potentially  autonomous  security  robots 
with  lethal  weaponry.16 

In  1990,  a  congressionally-mandated  joint  program 
office  was  established  to  better  coordinate  UGV  de¬ 
velopment  efforts  among  the  Army  and  the  Marine 
Corps,  a  relationship  that  is  part  of  the  overall  Joint 
Ground  Robotics  Enterprise  (JGRE)  construct.17  Into 
this  century,  after  the  9/11  attacks,  the  use  of  UGVs 
saw  a  rapid  expansion  for  Operation  IRAQI  FREEDOM 
(OIF)  and  Operation  ENDURING  FREEDOM  (OEF)  in 
response  to  warfighter  urgent  needs  for  systems  such 
as  PackBot,  TALON,  and  MARCBot.18  The  evolution 
of  these  systems  will  be  discussed  in  more  detail  in  the 
developmental  issues  section  of  this  Paper. 
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Contemporary  Applications. 

The  current  master  plan  to  coordinate  DoD  un¬ 
manned  system  efforts  is  described  in  considerable 
detail  by  the  Unmanned  Systems  Integrated  Roadmap, 
FY2013-2038,  a  document  approved  by  two  key  leaders 
within  DoD  acquisition  — Admiral  James  Winnefeld, 
Jr.,  the  Vice  Chairman  of  the  Joint  Chiefs  of  Staff;  and 
Frank  Kendall,  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology  and  Logistics  (USD(AT&L)). 
This  comprehensive  plan  addresses  strategic  planning 
and  policy,  technologies,  operating  environments,  lo¬ 
gistics  and  sustainment,  training,  and  international 
cooperation  for  unmanned  systems  applications  in 
the  domains  of  air,  land,  and  sea.19  Table  1  provides 
the  description  of  domain-based  unmanned  systems 
per  the  Integrated  Roadmap.  It  is  interesting  to  note 
that,  consistent  with  the  emphasis  on  air  systems, 
only  the  term  UAS  is  explicitly  defined  in  current  joint 
doctrine.20 


unmanned  aircraft  system 
(UAS) 

A  system  whose  components  include  the  necessary  equipment, 
network,  and  personnel  to  control  an  unmanned  aircraft. 

unmanned  ground  system 
(UGV) 

A  powered  physical  system  with  (optionally)  no  human  operator 
aboard  the  principal  platform,  which  can  act  remotely  to  ac¬ 
complish  assigned  tasks.  UGS  may  be  mobile  or  stationary,  can 
be  smart  learning  and  self-adaptive,  and  include  all  associated 
supporting  components  such  as  operator  control  units  (OCU). 

unmanned  maritime  system 
(UMS) 

UMS  comprise  unmanned  maritime  vehicles  (UMVs),  which 
include  both  unmanned  surface  vehicles  (USVs)  and  unmanned 
undersea  vehicles  (UUVs),  all  necessary  support  components, 
and  the  fully  integrated  sensors  and  payloads  necessary  to  ac¬ 
complish  the  required  missions. 

Table  1.  DoD  Domain-Based  Unmanned  System 
Descriptions.21 
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Interest  in  unmanned  systems  is  by  no  means  limit¬ 
ed  to  the  U.S.  military.  The  U.S.  Government  Account¬ 
ability  Office  (GAO)  has  conducted  numerous  studies 
that  examine  not  only  the  use  of  such  systems  among 
government  departments  and  agencies,  but  also  the 
implications  of  exporting  commercial  versions  of  some 
unmanned  systems  (or  their  related  technologies).22  In 
a  July  2012  report,  GAO  examined  the  issue  of  world¬ 
wide  proliferation  of  unmanned  aerial  vehicle  (UAV), 
noting  that  in  only  7  years,  "the  number  of  countries 
that  acquired  an  unmanned  aerial  vehicle  (UAV)  sys¬ 
tem  nearly  doubled  from  about  40  to  more  than  75. "23 
Figure  2  provides  a  graphical  depiction  of  the  magni¬ 
tude  of  the  worldwide  situation,  with  shaded  portions 
of  the  map  indicating  countries  that  operate  UAVs. 


Figure  2.  Countries  with  Unmanned  Aerial  Vehicles 
(as  of  December  2010).24 
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Contributing  to  this  proliferation  is  the  explosion 
of  the  domestic  UAS  market.  The  Association  for 
Unmanned  Vehicle  Systems  International  (AUVSI), 
an  industry  advocacy  group,  claims  that  the  current 
global  commercial  market  (manufacturers  and  appli¬ 
cations)  for  unmanned  systems  is  comprised  of  2,400 
platforms  and  900  companies  for  air  systems;  880  plat¬ 
forms  and  340  companies  for  ground  systems,  and  810 
platforms  and  340  companies  for  maritime  systems.25 
The  economic  potential  is  huge  for  a  UAV  industry  if 
and  when  it  is  fully  integrated  into  U.S.  airspace.  An 
AUVSI  report  from  March  2013  projects  the  potential 
for  70,000  new  jobs  within  the  first  3  years  of  inte¬ 
gration  with  an  economic  impact  of  more  than  $13.6 
billion,  and  this  is  further  projected  to  grow  to  over 
100,000  new  jobs  with  $82  billion  benefit  by  2025. 26  As 
with  many  high-technology  products,  unmanned  ve¬ 
hicles  may  have  dual-use  potential  for  both  commer¬ 
cial  and  government  purposes. 

Now  that  we  have  a  better  appreciation  for  the  di¬ 
verse  and  dynamic  context  of  unmanned  systems  in 
general,  let  us  now  examine  some  of  the  opportunities 
and  challenges  associated  with  their  military  applica¬ 
tions.  Consistent  with  DoD  guidance,  the  use  of  AWS 
in  any  part  of  the  range  of  military  operations  must 
include  the  opportunity  for  appropriate  human  judg¬ 
ment,  but  this  responsibility  also  extends  to  those  who 
design  and  develop  the  systems.27  The  next  section  ad¬ 
dresses  some  of  the  key  issues  facing  the  community 
that  takes  a  given  AWS  through  the  processes  from 
an  initial  requirement  or  concept  up  to  being  a  fully 
capable  field  asset. 
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DEVELOPMENTAL  ISSUES  RELATED 
TO  AUTONOMOUS  SYSTEMS 


What  does  DoD  hope  to  gain  from  the  use  of  AWS? 
The  Integrated  Roadmap  contends  that: 

unmanned  systems  will  be  critical  to  U.S.  operations 
in  all  domains  across  a  range  of  conflicts,  both  be¬ 
cause  of  their  capability  and  performance  advantages 
and  because  of  their  ability  to  take  greater  risk  than 
manned  systems.28 

Further,  it  cites  three  primary  forces  driving  DoD  ef¬ 
forts:  (1)  military  utility  of  unmanned  systems  as  dem¬ 
onstrated  in  combat  operations  in  Southwest  Asia; 
(2)  anticipated  budget  constraints  that  require  more 
affordable  technical  solutions;  and  (3)  changing  inter¬ 
national  security  environment,  especially  the  strategic 
shift  to  Asia-Pacific  Theater  and  the  resulting  anti-ac¬ 
cess/  area  denial  (A2/  AD)  challenges.29  Summing  it  all 
up,  the  vision  of  the  roadmap  states  simply,  "DoD  will 
develop  and  field  affordable,  flexible,  interoperable, 
integrated,  and  technologically  advanced  unmanned 
capabilities."30 

Toward  this  vision,  this  section  explores  two  ques¬ 
tions:  How  should  the  United  States  develop  auton¬ 
omy-related  applications  for  use  across  the  range  of 
military  operations?  What  technologies  are  impera¬ 
tive  to  the  development  of  AWS?  The  section  ad¬ 
dresses  these  questions  first  by  describing  the  current 
state-of-the-art  for  Army  UGV  systems.  It  then  exam¬ 
ines  technology  issues  in  decreasing  scope  of  applica¬ 
tion  —  from  those  for  unmanned  systems  in  general,  to 
those  related  to  the  use  of  autonomy,  to  those  specifi¬ 
cally  related  to  UGV  purposes.  The  section  ends  with 
a  brief  analysis  of  the  federal  budget  trends  for  AWS 
development. 
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Current  DoD  and  Army  Developmental  State 
of  the  Art  in  Autonomous  Systems. 

In  a  2012  report,  the  Defense  Science  Board  (DSB) 
examined  the  role  of  autonomy  in  DoD  systems  and 
concluded  that,  despite  the  proven  usefulness  of  un¬ 
manned  systems  in  operation,  "autonomy  technology 
is  being  underutilized  as  a  result  of  material  obstacles 
within  the  Department"  that  include  "poor  design 
[and]  lack  of  coordination  of  research  and  develop¬ 
ment  (R&D)  efforts."31  Further,  the  report  argues  that 
recent  development  of  unmanned  systems  was  large¬ 
ly  evolutionary,  moving  forward  based  on  combat 
experience  and  experimentation  in  Iraq  and  Afghani¬ 
stan.  The  report  does  argue  that  the  concept  of  armed 
UAVs,  which  combine  the  strike  and  intelligence, 
surveillance,  and  reconnaissance  (ISR)  missions  into 
a  single  platform,  is  a  revolutionary  new  capability. 
In  both  cases,  the  main  impetus  behind  the  systems 
was  an  operational  pull  to  support  missions  to  defeat 
improvised  explosive  devices  and  to  eliminate  high- 
value  targets.32 

The  push  to  acquire  UGV  system  rapidly  to  sup¬ 
port  OIF /  OEF  urgent  needs  led  to  widespread  pur¬ 
chase  of  commercial-off-the-shelf  (COTS)  equipment. 
Eventually,  over  7,000  UGV  of  many  different  designs 
were  purchased,  mostly  using  Overseas  Contingency 
Operations  (OCO)  funding.33  Table  2  depicts  the  evo¬ 
lution  in  functional  focus  of  ground  robotics  in  com¬ 
bat  from  162  systems  in  2004  to  over  7,000  by  2012.34 
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Year 

Number  of  Ground 
Robot  Systems 

Functional  Focus 

2004 

162 

-  No  single  vendor  could  produce  1 62 

-  Five  vendors,  multiple  configurations 

-  Joint  effort,  E0D  [explosive  ordnance  disposal]  focused 

2005 

1,800 

-  Robot’s  proven  ability  to  save  lives 

-  Expansion  beyond  E0D  mission  (Countermine,  Security) 

-  Agreements  with  AMC  [Army  Materiel  Command]  and  REF 
[Rapid  Equipping  Force] 

2006 

4,000 

-  Engineers  and  Infantry 

-  Route  clearance,  Explosive  detection  &  Weaponization 
development 

2007 

5,000 

-  Special  Forces  robot  applications  assessed 

-  Route  clearance,  Explosive  detection  &  Weaponization  on 
battlefield 

2008-09 

6,000 

-  Range  extension 

-  CBRNE  [Chemical,  Biological,  Radiological,  Nuclear,  Explo¬ 
sives]  detection 

-  Persistent  surveillance 

-  RC  [Remote  Control]  HMMWV 

-  More  capable  payloads 

-  Maneuver  elements 

2010-12 

7,000 

-  Military  Police 

-  Smaller  platforms 

-  Enhanced  battery  life 

-  Commonality 

-  More  capable  payloads 

2013-14 

>7,000 

-  “Plug  &  play”  capabilities 

-  Limited  autonomy 

-  Weaponization 

-  Increased  agility  and  dexterity 

-  Interoperability 

Table  2.  Evolution  of  Ground  Robotics  in  Combat.35 

In  parallel  to  the  ad  hoc  procurement  of  UGVs  for 
OIF / OEF,  a  methodical  and  coordinated  effort  to  in¬ 
corporate  unmanned  systems  was  ongoing  as  part  of 
the  Future  Combat  System  (FCS).  Initially  envisioned 
by  General  Eric  Shinseki  in  1999,  FCS  was  the  huge  pro¬ 
gram  ($200  billion)  that  integrated  18  systems  around 
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a  central  network.  Of  the  original  18  systems,  10  were 
to  be  unmanned:  four  different  classes  of  UAV;  three 
different  types  of  UGVs;  and  three  other  unmanned 
ground  systems  that  included  a  vertical  missile  launch 
system.  Although  the  program  achieved  its  Milestone 
B  of  the  DoD  acquisition  framework  in  2003,  FCS  was 
reduced  in  scope  and  eventually  cancelled  in  June 
2009.  Little  of  the  decade-long  development  effort  was 
salvaged.36 

As  OIF/OEF  closed  out,  the  Army  Deputy  Chief 
of  Staff,  Operations  and  Plans  (G-3/5/7)  directed  a 
bridging  strategy  to  sustain  select  existing  COTS  sys¬ 
tems  (primarily  Talon  and  Packbot  families)  for  up  to 
8  years  until  UGVs  are  available  via  traditional  acqui¬ 
sition  program  of  record  methodology.37  This  includes 
resetting  and  sustaining  almost  1,500  COTS  robots 
while  working  to  incorporate  interoperability,  chassis 
commonality,  and  payload  modularity  into  systems 
being  developed.38  The  anticipated  strategic  environ¬ 
ment  for  this  unmanned  system  development  is  large¬ 
ly  derived  from  capstone  DoD  studies  such  as  the 
Quadrennial  Defense  Review  as  well  as  priorities  from 
the  President  and  Congress.  Per  the  Integrated  Road¬ 
map,  the  explicit  strategic  trends  and  environmental 
characteristics  are: 

•  Reduction  in  federal  budgets; 

•  Operational  issues  will  be  more  complex; 

•  U.S.  military  forces  will  be  rebalanced  toward 
the  Pacific; 

•  Violent  extremism  will  continue  to  threaten 
U.S.  interests; 

•  Unmanned  technologies  will  continue  to  im¬ 
prove;  and, 

•  Cyber  domain  will  be  a  conflict  environment.39 
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The  Integrated  Roadmap  also  addresses  potential  im¬ 
plication  of  adversary  use  of  unmanned  systems, 
especially  in  support  of  A2/  AD  missions: 

Enemy  unmanned  systems  will  complicate  air,  ground, 
and  maritime  operations  by  adding  new  low-altitude, 
ground,  and  amphibious  threats  to  the  force  that  must 
be  countered.  This  concern  will  require  the  develop¬ 
ment  of  friendly  countermeasures,  including  tactics, 
techniques,  procedures,  and  training  that  enable  the 
force  to  operate  in  the  emerging  environment.40 

Given  these  myriad  challenges,  what  are  the  key  tech¬ 
nologies  necessary  to  support  the  development  of 
future  unmanned  systems? 

Key  Technologies  for  Unmanned  Systems. 

Determination  of  the  technology  foci  for  un¬ 
manned  systems  should  be  informed  by  the  expected 
capabilities  that  such  systems  will  fulfill  in  the  range 
of  military  operations.  The  framework  of  Joint  Capa¬ 
bility  Areas  (JCAs)  is  utilized  by  DoD  to  help  organize 
and  manage  this  process.  Currently,  there  are  four 
JCAs  envisioned  for  unmanned  systems  to  support: 
battlespace  awareness,  force  application,  protection, 
and  logistics.41  While  many  of  the  basic  requirements 
of  locomotion,  sensing,  navigation,  and  connectivity 
were  solved  at  the  prototype  level  in  the  1980s  and 
1990s,  and  further  refined  through  combat  experience 
in  the  21st  century,  there  remain  significant  technolog¬ 
ical  challenges  for  AWS  development.  The  Integrated 
Roadmap  narrows  this  to  nine  technology  areas  that 
are  presented  in  Table  3  along  with  summaries  of  the 
proposed  objectives  and  activities  for  each  area. 
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Technology  Areas 

Objectives  and  Activities 

Interoperability  and 
Modularity 

-  Develop  and  stabilize  standard  information  exchange  requirements  (lERs)  with  joint,  inter¬ 
agency,  intergovernmental,  and  multinational  (JIIM)  partners 

-  Meet  interoperability  requirements  such  as  those  specified  in  DoDD  5000.01  and  CJCSi 

621 2.01 F 

-  Create  interoperable  components,  subsystem,  and  payload  interface  to  enhance  modularity 
and  reduce  life  cycle  cost 

Communication 
Systems,  Spectrum, 
and  Resilience 

-  Pursue  platform  agnostic  command,  control,  communications,  and  computers  (C4)  infra¬ 
structure 

-  Simplify  plug-and-play  payload  interfaces  to  lower  costs  and  enhance  ability  to  update, 
modify,  upgrade,  and  link 

-  Utilize  globally  available  C4  enterprise  capabilities  (such  as  data  centers  and  distribution 
nodes) 

Security:  Research 
and  Intelligence/ 
Technology 

Protection  (RITP) 

-  Use  layered  application  of  protective  measures  to  prevent  compromise  of  critical  informa¬ 
tion  and  technology 

-  Assess  system  vulnerabilities  and  threats  as  early  as  reasonable  in  the  development 
process 

-  Provide  unmanned  systems  with  the  ability  to  remotely  and  autonomously  render  data  at 
rest  unrecoverable  by  an  adversary 

-  Create  classification  guidelines  that  transcend  organizational  cultures  and  build  trust 
between  Intelligence  Community  agencies  and  mission  partners 

Persistent  Resilience 

-  Reduce  the  size,  weight,  and  power  consumption  of  unmanned  systems 

-  Achieve  reliability,  availability,  and  maintainability  (RAM)  performance  sufficient  for  long- 
duration  missions  (such  as  single  unmanned  vehicle  ISR  for  periods  exceeding  24  hours) 

-  Utilize  active  warning  and  self-protection  subsystems  as  well  as  passive  measures  to 
enhance  survivability 

-  Pursue  advanced  structures,  material,  and  propulsion  solutions 

Autonomy  and 
Cognitive  Behavior 

-  Move  unmanned  systems  from  executing  preprogrammed  plans  to  performing  missions 
with  dynamic  tasks 

-  Develop  ability  for  unmanned  systems  to  modify  strategies  and  seif-directed  behavior 
necessary  to  achieve  a  human-directed  goal 

-  Incorporate  key  enablers  such  as  dynamic  mission  planning,  precise  position,  navigation, 
and  timing  (PNT),  and  appropriate  machine  rules  of  engagement  for  utilizing  processed 
information  and  mitigating  lost  links 

Weaponry 

-  Develop  and  standardize  weapons  specifically  designed  for  use  on  unmanned  systems 

-  Design  weapons  with  multiple  modes  of  operation  that  include  the  ability  for  scalable  ef¬ 
fects  (similar  in  concept  to  nuclear  “dial-a-yield”) 

-  Utilize  advanced  weapons  materials  and  nanoenergetics  to  increase  performance  and 
reduce  weight 

Table  3.  DoD  Key  Areas  for  the  Technological 
Advancement  of  Unmanned  Systems.42 
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Tackling  these  technologies  in  a  concerted  fashion 
may  yield  dramatic  improvements  that  facilitate  mis¬ 
sion  performance,  endurance,  reliability,  and  synchro¬ 
nization  at  reduced  levels  of  human  risk  and  logistics 
burden.  When  considering  development  of  lethal 
AWS,  the  areas  of  weaponry  and  autonomy  and  cog¬ 
nitive  behavior  merit  further  insight.  The  evolution  of 
weaponized  AWS  has  been  largely  an  ad  hoc  process 
that  adapted  unmanned  platforms  to  use  available 
munition  systems.  This  has  been  proven  effective  for 
combinations  such  as  Hellfire  missiles  on  Predator 
UAVs.  But  there  is  great  potential  for  optimization 
at  the  system-of-systems  level  if  the  weaponry  is  de¬ 
signed  specifically  for  unique  features  of  a  given  un¬ 
manned  platform.  In  this  way,  the  munition  system 
may  be  better  suited  to  operate  within  the  unmanned 
systems'  environment  (including  optimized  size  and 
shape),  to  exchange  mission  data,  and  to  be  inter¬ 
changeable  within  classes  of  unmanned  systems.43 

For  autonomy  and  cognitive  behavior,  the  aim  can 
be  stated  simply  as  "the  future  of  autonomous  sys¬ 
tems  is  characterized  as  a  movement  beyond  autono¬ 
mous  mission  execution  to  autonomous  mission  perfor¬ 
mance A44  In  other  words,  unmanned  systems  should 
strive  to  emulate  appropriate  cognitive  behavior  in 
the  battlespace,  just  as  the  brain  of  a  military  opera¬ 
tor  must  adapt  to  the  inevitable  changes  that  transpire 
as  they  move  from  the  planning  cell  to  "boots  on  the 
ground"  as  well  as  anticipate  future  change.45  Pursu¬ 
ing  the  concept  of  autonomy  requires  its  own  subset 
of  supporting  technologies. 
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Technologies  that  Facilitate  Autonomy. 

How  does  one  break  down  the  complex  concept  of 
autonomy  into  practical  elements  that  can  be  realized 
through  use  of  the  proper  technology?  A  framework 
developed  by  the  Office  of  Naval  Research  (ONR)  ex¬ 
amines  how  varying  levels  of  autonomy  may  be  re¬ 
quired  to  enable  mission  accomplishment  as  part  of  an 
iterative  process  that  exercises  an  AWS  ability  to  form 
a  world  model,  the  ability  to  reason,  and  the  ability  to 
alter  actions.  When  actions  are  altered  by  the  AWS, 
the  world  model  changes  and  the  cycle  repeats.46  In  an 
unrelated  effort,  in  a  2012  task  force  report,  the  DSB 
identified  "six  key  areas  in  which  advances  in  auton¬ 
omy  would  have  significant  benefit  to  the  unmanned 
system"47  that  are  summarized  in  Table  4.  These  six 
areas  correlate  well  to  the  ONR  framework,  with  per¬ 
ception  technology  linked  to  determining  a  world 
model;  planning  and  learning  technologies  linked  to 
the  ability  to  reason;  and  human-robot  interaction, 
natural  language  understanding,  and  multi-agent  co¬ 
ordination  linked  to  the  ability  to  alter  actions.  The 
DSB  study  also  examined  the  state  of  the  art  of  each 
of  these  six  areas  and  provided  an  assessment  of  gaps 
that  should  be  addressed  to  improve  the  application 
of  autonomy  for  DoD  systems. 
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Technology  Areas 

Application  for  Autonomy  in  Unmanned  Systems 

Perception 

Perception  is  essential  for  autonomy,  for  enabling  the  unmanned  vehicle  to  achieve  reach  (e.g., 
navigate  through  environments  and  manipulate  obstacles)  and  for  using  reach  to  meet  mission 
objectives,  either  for  a  platform  (e.g.,  collecting  sensor  data,  applying  kinetic  weapons,  defeat¬ 
ing  lEDs)  or  for  the  battlespace. 

Planning 

Planning  is  the  process  of  computing  a  sequence  or  partial  order  of  actions  that  change  the 
world  from  the  current  state  to  a  desired  state,  or  in  DoD  terminology,  a  plan  is  a  course  of  ac¬ 
tion  designed  to  achieve  mission  objectives  while  minimizing  resource  utilization.  The  process 
relies  on  two  key  components:  1)  a  representation  of  actions,  descriptions  of  conditions  in  the 
world  and  objectives/resource  optimization  criteria  and  2)  algorithms  for  computing  action 
sequences  and  assigning  resources  to  the  actions  so  as  to  conform  to  the  hard  constraints  of 
the  problem  (e.g.,  vehicle  limitations  in  terms  of  terrain  and  speed)  while  optimizing  the  soft 
constraints  (e.g.,  minimizing  the  total  mission  time  or  personnel  use). 

Learning 

Machine  Learning  has  become  one  of  the  most  effective  approaches  to  developing  intelligent, 
autonomous  systems.  Automatically  inducing  knowledge  from  data  has  generally  been  found  to 
be  more  effective  than  manual  knowledge  engineering.  Development  of  state-of-the-art  systems 
in  computer  vision,  robotics,  natural  language  processing  and  planning  now  rely  extensively  on 
automated  learning  from  training  data.  Mining  large  amounts  of  real-world  data  to  find  reliable 
patterns,  generally  results  in  more  accurate  and  robust  autonomous  systems  than  manual 
software  engineering.  This  also  allows  a  system  to  automatically  adapt  to  novel  environments 
from  actual  experience  operating  in  these  situations. 

Human-Robot 

Interaction 

Human-robot  interaction  (HRI)  is  a  relatively  new,  multi-disciplinary  field  that  addresses  how 
people  work  or  play  with  robots  versus  computers  or  tools.  This  is  a  subset  of  the  larger  field  of 
human-system  interaction,  as  the  focus  is  on  bi-directional,  cognitive  interactions  in  which  the 
robot  is  a  physically  situated  agent  operating  at  a  distance  from  the  user,  versus  a  computer  or 
autopilot,  thus  leading  to  significant  distinctions.  In  order  to  be  consistent  with  the  scientific  lit¬ 
erature,  the  term  HRI  will  be  used  generally,  but  UxV  [unmanned  vehicle]  will  serve  to  describe 
the  specific  form  of  robot. 

Natural  Language 
Understanding 

Natural  language  is  the  most  normal  and  intuitive  way  for  humans  to  instruct  autonomous 
systems;  it  allows  them  to  provide  diverse,  high-level  goals  and  strategies  rather  than  detailed 
teleoperation.  However,  understanding  human  language  is  difficult  since  it  is  inherently  ambigu¬ 
ous,  and  context  must  be  used  to  infer  the  intended  meaning.  Therefore,  building  autonomous 
systems  that  can  follow  English  instructions  as  well  as  human  speech  is  a  very  difficult  techni¬ 
cal  challenge.  Therefore,  traditional  graphical  user  interfaces  (GUIs)  are  frequently  a  more 
effective  approach  to  communicating  with  computing  systems.  However,  in  many  situations 
(e.g.,  when  the  user’s  hands  are  otherwise  engaged),  language  is  a  very  desirable  mode  of 
communication. 

Multi-agent 

Coordination 

Multi-agent  coordination  is  a  term  that  is  broadly  applied  to  accomplishing  a  task  that  is 
distributed  over  multiple  robots,  software  agents  or  humans.  Each  agent  is  considered  to  have 
some  degree  of  individual  autonomy,  and  the  coordination  may  either  emerge  from  the  agents 
interacting  or  negotiating  with  each  other  directly  (distributed  coordination)  or  be  explicitly 
directed  by  a  planner  (centralized  coordination).  Regardless  of  the  coordination  scheme,  the 
distribution  of  an  activity  across  multiple  agents  implies  that  coordination  schemes  must  ad¬ 
dress  synchronization  of  the  agents  with  each  other  and  to  dynamically  changing  aspects  of  the 
environment  or  mission. 

Table  4.  DSB  Key  Areas  for  Advancement 
in  Autonomy  to  Benefit  Unmanned  Systems.48 
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In  practical  terms,  any  AWS  that  embraces  the 
technologies  in  Table  4  in  its  design  must  also  meet 
strict  operational  effectiveness  and  suitability  test  pa¬ 
rameters  as  certified  by  the  Director  of  Operational 
Test  and  Evaluation.49  For  Army-specific  applications, 
UGVs  will  need  to  support  the  traditional  "shoot- 
mo  ve-communic  ate"  technique  of  ground  forces,  but 
they  must  do  so  in  "environments  [that]  could  include 
being  thrown  or  launched,  climbing  hills  or  stairs, 
and  hopping  and  landing  upright."50  Independent  of 
the  DSB  findings,  the  Army  science  and  technology 
(S&T)  community  identified  the  need  for  enhanced 
capabilities  in  five  areas:  adaptive  tactical  reasoning; 
focused  situational  awareness;  safe,  secure,  and  adap¬ 
tive  movement;  efficient  proactive  interaction  with 
humans;  and  interaction  with  the  physical  world.51 

To  pursue  these  capabilities,  the  Army  Research 
Laboratory  formed  the  Robotics  Collaborative  Tech¬ 
nology  Alliance  (RCTA)  as  a  consortium  of  nine  gov¬ 
ernment,  industrial,  and  academic  institutions  to  ad¬ 
dress  research  and  development  relevant  to  future 
military  unmanned  ground  vehicles.52  The  RCTA 
Program  Plan  simplified  the  title  of  the  five  enhanced 
capability  areas  to:  think-look-move-talk-work.  Table 
5  summarizes  the  vision  and  challenges  of  each  ca¬ 
pability  area.  The  RCTA  plan  goes  on  to  identify  five 
primary  cross-cutting  technical  barriers  which  are 
compared  across  the  think-look-move-talk-work  par¬ 
adigm  to  identify  the  fundamental  research  thrusts 
that  the  consortium  will  pursue.53 
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Enhanced 

Vision  and  Challenges 

Capability  Area 

Adaptive  Tactical 
Reasoning 

-  Vision:  robots  understand  the  concept  of  a  mission  or  task,  including  stages  of  progress  and 

measures  of  success. 

“Think” 

-  Adaptive  tactical  reasoning  requires  both  declarative  and  procedural  knowledge  with  which  to 
reason.  Neither  exists  in  current  systems,  which  generally  have  no  data  structures  for  mission 
level  information.  Tactical  reasoning  also  requires  some  kind  of  model  of  the  other  members 

of  the  team,  both  human  and  robot,  so  that  reasonable  predictions  of  expected  behavior  can 

be  made. 

Focused  Situational 

Awareness 

-  Vision:  future  autonomous  ground  systems  maintain  situational  awareness  (SA)  that  is 
relevant  to  the  current  task  and  the  larger  mission. 

“Look” 

-  Focused  SA,  requires  a  semantic/cognitive  description  of  the  robot’s  environment  that 
current  systems  do  not  have.  SA  also  requires  a  sense  of  salience,  what  is  important  based 
on  a  shared  understanding  among  teammates.  Better  learning  is  needed  to  develop  a  more 
human-like  hierarchical  understanding  of  object  categories  in  the  first  place  as  well  as  to  refine 
perception  capabilities  in  the  field. 

Efficient  Proactive  In¬ 
teraction  with  Fiumans 

-  Vision:  robots  interact  with  each  other  and  especially  with  Soldiers  in  an  efficient  and  proactive 
way  relevant  to  the  evolving  situation. 

“Talk” 

-  Existing  robotic  systems  are  notoriously  opaque  and  distrusted.  They  cannot  explain  what 
they  are  doing,  primarily  because  they  do  not  have  meta-cognition;  in  other  words,  they  do  not 
have  a  model  of  their  own  behavior.  Current  systems  also  lack  the  ability  to  understand  human 
(i.e.,  semantic)  communication  of  orders  or  other  information 

Safe,  Secure,  and 
Adaptive  Movement 

-  Vision:  robots  that  move  on  orders  or  their  own  initiative  from  one  tactical  position  to  the 
next  with  little  or  no  reliance  on  metric  inputs  such  as  GPS.  They  can  move,  as  Soldiers  do,  to 
semantically  described  locations  (e.g.,  “third  building  on  the  left  after  the  next  intersection”). 

“Move” 

-  Current  systems  have  insufficient  descriptions,  or  models,  of  the  world  in  which  the  robot  is 
moving.  Useful  movement  is  also  hampered  by  the  lack  of  task  or  mission  context  so  that  a 
robot  may  persist  in  trying  to  reach  a  particular  location  that  is  not  needed  for  the  mission. 
Robots  also  need  to  be  able  to  move  in  crowded  and  unpredictable  environments,  where  exist¬ 
ing  algorithmic  approaches  are  probably  intractable  but  new  learning  approaches  may  work. 

Interaction  with  the 

Physical  World 

-  Vision:  robots  are  able  to  observe  objects  at  close  quarters  to  enable  3D  interaction  with  them. 
They  pick-up  and  move  objects,  either  upon  semantic  direction  or  their  own  initiative. 

“Work” 

-  The  above  four  capabilities  (think-iook-move-talk)  largely  enable  the  performance  of  the 
main  goal  of  the  mission  -  the  “work”  the  robot  is  to  do.  The  work  most  often  involves  direct 
physical  interaction  with  the  world:  entering  and  searching  a  building  or  vehicle,  loading  and 

delivering  supplies,  inspecting  a  suspected  IED,  etc.  There  is  generally  great  uncertainty  about 
the  objects  with  which  the  robot  is  attempting  to  interact. 

Table  5.  Research  and  Development  Focus  for 
Military  UGVs.54 
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The  fact  that  the  ONR,  DSB,  and  Army  S&T  studies 
identified  similar  sets  of  technology  areas  and  goals  for 
AWS  provides  confidence  that  their  efforts  are  in  the 
right  direction  and  that  they  can  support  each  other. 
However,  any  plan  to  implement  technology  comes 
with  a  price  tag  that  must  compete  in  an  increasingly 
contentious  federal  budget. 

Budget  Resources. 

The  DoD  Integrated  Roadmap  included  a  projec¬ 
tion  of  unmanned  systems  budgets  from  Fiscal  Year 
(FY)  2014  to  FY  2018  based  on  information  available 
at  its  publication,  included  herein  as  Figure  3.  Clear¬ 
ly,  the  budget  heavily  favors  air-based  systems  with 
their  projected  portions  accounting  for  94.2  percent  of 
the  total  development  costs  and  91.5  percent  of  total 
procurement  costs.  Of  course,  some  of  these  costs  in¬ 
clude  unmanned  air  systems  for  the  Army,  Navy,  and 
Marines  as  well  as  for  the  Air  Force.  In  stark  contrast 
is  the  lesser  budget  priority  for  unmanned  ground 
systems,  which  account  for  less  than  1  percent  of  to¬ 
tal  development  costs  and  less  than  2  percent  of  total 
procurement  costs. 
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FYDP 

2014 

12015  | 

2016 

|2017  | 

2018 

Total 

Air 

RDTE 

1189.4 

1674.0 

1521.4 

1189.4 

1087.9 

6662.2 

Proc 

1505.5 

2010.2 

1843.5 

1870.7 

2152.8 

9382.7 

OM 

1080.9 

1135.2 

1102.7 

1156.9 

1178.5 

5654.1 

Domain  Total 

3775.9 

4819.4 

4467.6 

4217.0 

4419.3 

21699.1 

FYDP 

2014 

2015 

2016 

2017 

2018 

Total 

Ground 

RDTE 

6.5 

19.1 

13.6 

11.1 

10.6 

60.9 

Proc 

6.5 

27.9 

30.7 

42.6 

55.4 

163.1 

OM 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

Domain  Total 

13.0 

47.0 

44.3 

53.7 

66.0 

223.9 

FYDP 

2014 

2015 

2016 

2017 

2018 

Total 

Maritime 

RDTE 

62.8 

54.8 

66.1 

81.0 

87.2 

351.9 

Proc 

104.0 

184.8 

160.1 

158.1 

101.1 

708.2 

OM 

163.4 

170.3 

182.4 

190.5 

193.6 

900.2 

Domain  Total 

330.2 

409.8 

408.6 

429.7 

381.8 

1960.2 

FYDP 

2014 

2015 

2016 

2017 

2018 

Total 

All 

Unmanned 

Systems 

RDTE 

1,258.7 

1,747.9  1,601.1 

1,281.5 

1,185.7 

7,075.0  I 

Proc 

1,616.0 

2,222.9  2,034.3 

2,071.4 

2,309.3 

10,253.9 

OM 

1,244.3 

1,305.4  1,285.1 

1,347.4 

1,372.1 

6,554.3  | 

Domain  Total 

4,119.1 

5,276.2 

4,920.5 

4,700.4 

4,867.1 

23,883.2  | 

Figure  3.  DoD  Unmanned  Systems  Funding  per 
Integrated  Roadmap.55 

While  there  is  insufficient  information  to  repro¬ 
duce  the  method  by  which  Figure  3  was  tabulated, 
we  can  compare  some  of  its  projected  values  to  those 
recently  submitted  as  part  of  the  FY  2016  DoD  budget 
request.  Procurement  of  UAV  dominates  the  requests 
for  unmanned  systems  and  it  includes  $960  million  for 
research,  development,  test  and  evaluation  (RDT&E) 
as  well  as  $1.87  billion  for  procurement.  The  procure¬ 
ment  funding  buys  51  UAVs:  15  MQ-1  Gray  Eagles 
(based  on  Predator)  for  the  Army;  29  MQ-9  Reapers 
for  the  Air  Force;  3  MQ-4C  Tritons  (based  on  Global 
Hawk)  for  the  Navy;  and  4  RQ-21  Blackjacks  (based 
on  Shadow)  for  the  Marine  Corps.56  The  Army  uses 
the  Gray  Eagle  as  "a  dedicated,  assured,  multi-mis- 
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sion  Unmanned  Aircraft  System  (UAS)  fielded  to  all 
ten  Army  Divisions  to  support  the  commander's  com¬ 
bat  operations."57 

Funding  for  various  unmanned  system  program 
elements  are  spread  through  the  Army's  FY  2016  bud¬ 
get  request,  and  it  is  beyond  the  scope  of  this  Paper 
to  delve  into  all  the  details  therein.  However,  in  addi¬ 
tion  to  the  UAV  and  related  costs  already  identified, 
there  are  two  noteworthy  items  for  unmanned  ground 
systems.  First  is  the  request  for  $40.4  million  RDT&E 
in  support  of  the  Tactical  Unmanned  Ground  Vehicle 
(Program  Element  0604641 A)  which  is  described  as 
"man-packable,  miniature  (<25-lbs),  highly  mobile, 
unmanned  robotic  system  with  advanced  sensors/ 
mission  modules  for  dismounted  Soldiers."58  The  pro¬ 
gram  element  includes  the  ability  to  evaluate  readily 
available  hardware  such  as  COTS  to  support  emerging 
warfighter  requirements  as  part  of  the  Robot  Enhance¬ 
ment  Program.59  Second  is  an  indication  of  the  Army's 
intent  to  pursue  ground  AWS  that  can  apply  lethal 
capabilities  listed  under  Program  Element  0603827A, 
Soldier  Systems- Advanced  Development,  Project  S54, 
Small  Arms  Improvement: 

FY16  New  Start  Armaments  for  Robots:  Will  initiate 
the  intelligence/  networking  and  weapons  design  and 
functions  for  a  man-in-the-loop,  small  caliber  defen¬ 
sive  armaments  system  on  an  unmanned  ground  ve¬ 
hicle  including  the  Warfighter/Robot  interface.60 

No  further  details  are  available,  but  this  would  ap¬ 
pear  to  support  the  intent  of  the  Integrated  Roadmap 
technology  area  of  weaponry  development  specific  to 
unmanned  platforms. 

The  budget  challenges  are  significant  as  FY  2016 
and  beyond  must  deal  with  the  loss  of  OCO  funding 
as  well  as  possible  impacts  from  sequestration.  Spend- 


23 


ing  for  UGVs  must  support  the  bridging  strategy  to 
divest  older  systems  and  limit  sustainment  costs  on 
the  systems  selected  to  remain  in  the  inventory.  This 
will  leave  little  funding  available  to  develop,  procure, 
and  deploy  future  unmanned  systems,  thus  it  is  im¬ 
portant  to  understand  how  these  systems  may  be  used 
in  military  operations  to  ensure  investments  are  made 
with  prudence. 

OPERATIONAL  ISSUES  RELATED  TO  USE  OF 
AUTONOMOUS  SYSTEMS 

One  could  argue  that  the  operational  use  of  AWS 
thus  far  has  been  viewed  largely  as  an  ends  to  a  means 
focused  on  a  specific  mission,  such  as  UGVs  support¬ 
ing  counter-improvised  explosive  device  (C-IED)  ef¬ 
forts.  But  the  Army  Research  Laboratory  RCTA  over¬ 
view  brief  offered  a  different  vision  for  unmanned 
systems  — "A  Paradigm  shift— from  Tool  to  Team 
Member"61  This  section  addresses  two  questions  re¬ 
lated  to  such  a  vision:  What  can  be  achieved  via  au¬ 
tonomy  within  the  range  of  military  operations?  What 
missions  can  be  enhanced  by  incorporating  AWS? 

Mission  Parameters. 

The  parameters  for  potential  mission  areas  for 
AWS  can  be  found  in  each  of  four  JCAs  identified  in 
the  Integrated  Roadmap.  For  battlespace  awareness, 
AWS  may  extend  mission  endurance  and  provide 
persistent  ISR  in  all  operational  domains  that  could  be 
readily  shared  with  similar  systems  to  further  enhance 
their  effectiveness.  Force  application  capabilities  have 
already  been  proven  in  combat  by  UAVs  conducting 
offensive  operations  against  high-value  targets.  UGVs 
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are  also  projected  to  have  lethal  applications,  such 
as  armed  reconnaissance,  as  well  as  nonlethal  opera¬ 
tions,  such  as  crowd  control.  For  protection  capabili¬ 
ties,  unmanned  systems  can  assume  risky  missions, 
such  as  firefighting  and  decontamination,  as  well  as 
continue  to  refine  methods  for  handling  sophisticated 
high  explosives.  For  logistics,  unmanned  systems  can 
facilitate  transportation  and  resupply  tasks  as  well  as 
support  routine  maintenance,  such  as  inspections  and 
refueling.62 

Regarding  future  UGV  missions,  the  DSB  assesses 
that  both  the  Army  and  Marine  Corp  are  interested 
in  achieving  lower  risk  to  humans;  accessibility  to 
areas  not  suitable  for  humans;  enhanced  sensor  ca¬ 
pability  and  mobility;  and  battle  formations  that  pur¬ 
posely  combine  warfighters  with  AWS  units.63  In  the 
same  study,  the  DSB  asserted  that  the  application  of 
unmanned  systems  this  century  has  been  ad  hoc  in 
nature: 

Due  to  the  understandable  pressures  of  war,  un¬ 
manned  systems  were  often  fielded  before  CONOPS 
were  fully  developed  or  understood;  deployment 
support  structures  (sustainment,  service  structures, 
etc.)  were  immature;  and  the  lack  of  understanding 
or  validating  (testing)  maturity  to  support  tactical  and 
operational  challenges  in  remote  theaters  have  further 
complicated  progress.64 

To  address  the  warfighter  needs  for  unmanned  sys¬ 
tems  that  better  integrate  into  the  full  range  of  military 
operations,  let  us  consider  a  more  deliberate  approach 
to  their  integration  into  existing  force  structures. 
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Integration  into  Force  Structure. 

DoD  Directive  3000.09  requires  commanders  of  the 
combatant  commands  to  integrate  AWS  into  their  op¬ 
erational  mission  planning  and  also  charges  them  to 
"identify  warfighter  priorities  and  operational  needs 
that  may  be  met  by"  AWS  through  established  pro¬ 
cedures  overseen  by  the  Chairman,  Joint  Chiefs  of 
Staff.65  The  Integrated  Roadmap  posits  that  this  man¬ 
date  can  lead  to  systems  that  perform  beyond  mere 
substitution  for  manned  systems,  thus  allowing  new 
formations  and  tactics  that  are  more  agile  and  maneu¬ 
verable.  It  also  suggests  that  unmanned  systems  could 
be  used  as  low-cost  disposable  assets  that  expect  at¬ 
trition,  a  concept  that  reverses  the  current  focus  on 
procuring  complex  and  expensive  systems  that  often 
require  risk-adverse  planning.66  But  how  does  one  de¬ 
termine  the  best  types  of  unmanned  systems  to  pro¬ 
cure?  And  what  is  the  best  way  to  blend  these  systems 
with  human  warfighters  into  a  cohesive  force? 

As  discussed  earlier,  FCS  was  the  Army's  modern¬ 
ization  program  until  its  cancellation  in  2009.  But  in 
the  years  prior  to  this,  the  program  made  significant 
strides  toward  answering  these  force  integration  ques¬ 
tions.  The  FCS  Test  Master  Plan  addressed  not  only 
the  individual  capabilities  of  its  eight  manned  and 
10  unmanned  vehicles,  but  also  how  the  entire  fam¬ 
ily  of  systems  worked  together  to  accomplish  a  given 
mission.67  Also,  the  FCS  program  formed  a  unique 
Evaluation  Brigade  Combat  Team  at  Fort  Bliss,  Texas, 
to  support  early  operational  assessments  and  testing 
of  FCS  elements  as  well  as  the  complete  system  com¬ 
prised  of  soldiers  working  together  with  manned  and 
unmanned  platforms  on  the  ground  and  in  the  air.68 
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Currently,  the  Army  UGV  Campaign  Plan  for 
modernization  organizes  systems  into  four  classes  of 
vehicles  (CoV):  Soldier  transportable,  vehicle  trans¬ 
portable,  self-transportable,  and  applique.69  The  de¬ 
scriptions  of  the  CoV  and  system  types  within  each 
category  are  presented  in  Table  6.  Even  during  mod¬ 
ernization,  some  traditions  remain  and  the  Army  prac¬ 
tice  of  using  mules  is  updated  with  the  Common  Mo¬ 
bility  Platform/Multi-Mission  UGV  envisioned  "as 
vehicles  to  serve  as  robotic  'mules'  to  take  on  multiple 
soldiers'  loads."70  The  top  priority  for  technology  ar¬ 
eas  to  support  the  UGV  Campaign  Plan  was  assessed 
to  be  autonomy,  with  envisioned  mission  applications 
for: 


area  clearance,  route  clearance  (marks  and  detects), 
convoy,  soldier  follower,  manned/ unmanned  team¬ 
ing,  situational  awareness  and  navigation  in  a  [global 
positioning  system]  denied  environment.  Also  for 
route  detection,  planning,  and  maneuver  capabilities 
over  soldier  passable  terrain.71 

Emphasizing  autonomy  development  would  help 
mitigate  two  of  the  primary  challenges  assessed  by 
the  DBS  for  UGVs  in  combat  operations:  "negotiating 
terrain  and  obstacles  on  the  battlefield,  and  perform¬ 
ing  kinetic  operations  within  the  Rules  of  Engagement 
(ROE)."72 
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Class  of  Vehicles  (CoV) 

CoV  Description  and  System  Types 

Soldier  Transportable 

-  UGV  system  with  weight  not  exceeding  35  pounds  and  with  forms  that  allow  them  to 
be  carried  by  Soldiers  or  Marines  for  extended  periods  of  time  over  varying  terrains. 

-  The  majority  of  soldier  transportable  systems  are  used  for  surveillance,  reconnais¬ 
sance  missions,  and  standoff  IED  detection  and  defeat. 

-  Continued  advancements  in  antenna  design,  autonomy,  miniaturization,  power 
sources,  and  control  mechanisms  are  required  in  order  for  these  capabilities  to  be 

fully  realized. 

-  System  types:  Crew  Served  Bot ;  Small  Bot;  Micro  Bot ;  Nano  Bot 

Vehicle  Transportable 

-  Vehicle  transportable  unmanned  systems  are  heavier  and  require  a  prime  mover  for 
transportation  to  and  from  a  mission. 

-  Future  requirements  for  vehicle  transportable  systems  are  expected  to  include  more 

advanced,  reliable  and  autonomous  area  and  route  clearance  robotic  vehicles  as  well 

as  humanoid  like  systems. 

-  Continued  advancements  in  autonomy  (to  include  intelligence  understanding  and 
decision  making),  power  systems,  and  enhanced  mine  detection  and  neutralization 
techniques  and  methods  are  needed. 

-  System  types:  Mounted  or  Towed;  Armed;  Humanoid 

Self  Transportable 

-  Self  transportable  systems  can  move  under  their  own  power,  up  to  road  march 
speeds,  without  assistance  from  a  prime  mover  or  other  sources. 

-  Self  transportable  systems  are  not  manned  systems  with  applique  kits  applied,  rather 
they  are  systems  that  have  been  developed  explicitly  as  unmanned  vehicles. 

-  Continued  advancements  in  autonomy,  sensors  and  sensor  fusion  are  required  in 

order  for  these  capabilities  to  be  realized. 

-  System  types:  Soldier  Follower — Infantry  Brigade  Combat  Team  (IBCT);  Medium 
Wingman — Striker  Brigade  Combat  Team  (SBCT);  Heavy  Wingman — Heavy  Brigade 
Combat  Team  (HBCT);  Squad  Member. 

Applique 

-  Applique  COVs  are  systems  that  can  be  used  to  convert  fielded  and  future  manned 

systems  into  unmanned  systems. 

-  These  systems  are  envisioned  as  ‘kits’  that  include  all  the  hardware  (sensors,  cables, 
actuators,  control  station,  etc.)  and  software  required  to  fully  operate  and  monitor  the 
selected  vehicle  remotely. 

-  Key  advancements  are  required  in  the  areas  of  autonomy,  processing,  size,  weight 
and  power  (SWaP),  sensor  development,  and  sensor  fusion  for  this  CoV  to  be  fully 

matured. 

-  System  types:  Remote  Operation;  Supervised  Autonomy;  Full  Autonomy;  Exoskeleton 

Table  6.  Army  Unmanned  Ground  Vehicle 
(UGV)  Campaign  Plan.73 
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The  DoD  Roadmap  (2013)  makes  it  clear  that  force 
integration  for  unmanned  systems  must  not  be  con¬ 
fined  to  single-domain  applications.  Rather,  it  pro¬ 
mulgates  the  development  of  Manned-Unmanned 
Systems  Team  (MUM-T)  as  an  essential  part  of  the 
strategic  shift  to  the  Asia-Pacific  region.  The  MUM-T 
force  would  be  smaller  and  more  agile,  thus  providing 
key  capabilities  to  address  A2/  AD  challenges  such  as: 

•  Defeating  explosive  ground  surface,  sub-sur¬ 
face  (tunnel),  and  sea  hazards  from  greater 
standoff  distances; 

•  Assuring  mobility  to  support  multiple  points  of 
entry; 

•  Enabling  movement  and  maneuver  for  project¬ 
ing  offensive  operations; 

•  Establishing  and  sustaining  the  shore  lines  of 
communications  required  to  follow  forces  and 
logistics; 

•  Protecting  austere  combat  outposts; 

•  Providing  persistent  surveillance  to  detect  and 
neutralize  threats  and  hazards  within  single-  to 
triple-canopy  and  urban  terrain.74 

To  develop  and  refine  the  MUM-T  concept,  the 
Army's  Program  Executive  Office-Aviation  estab¬ 
lished  the  Manned  Unmanned  Systems  Integration 
Capability  (MUSIC)  Exercise  program.  The  first  exer¬ 
cise,  MUSIC  I,  was  conducted  in  September  2011  at 
Dugway  Proving  Ground,  Utah.  In  addition  to  MUM- 
T  capabilities,  the  exercise  also  tested  ground  control 
stations  and  remote  video  terminals.  Lessons  learned 
from  MUSIC  I  were  used  by  the  product  office  to  help 
improve  usability,  reliability,  integration,  and  config¬ 
uration  control  of  future  systems.75 
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Success  in  the  MUM-T  force  depends  not  only  on 
the  interactions  between  human  and  machines,  but 
also  on  the  interoperability  among  the  machines.  As  an 
example  of  ongoing  effort  to  improve  interoperability, 
consider  the  DoD  Unmanned  Aircraft  Systems  Task 
Force  (UAS  TF)  that  was  established  in  April  2010  in  a 
charter  signed  by  USD(AT&L)  and  the  Deputy  Secre¬ 
tary  of  Defense  to  address  many  issues  in  this  area.76 
The  UAS  TF  is  divided  into  four  principal  integrated 
product  teams  (IPTs):  Interoperability;  Airspace  In¬ 
tegration;  Frequency  and  Bandwidth;  and  Logistics 
and  Sustainment.  The  Task  Force  also  has  Advisory 
Groups  for  Research  and  Development  and  Payloads 
and  Sensors.77  For  the  critical  area  of  interoperability, 
the  Interoperability-IPT  "continues  to  address  the  29 
prioritized  Joint  interoperability  capability  gaps  iden¬ 
tified  in  the  approved  Unmanned  Interoperability  Ini¬ 
tiative  (UI2)  Capability  Based  Assessment  (CBA)."78 

Doctrine. 

DoD  Directive  3000.09  mandates  the  development 
of  doctrine  and  tactics,  techniques,  and  procedures 
(TTPs)  for  AWS  applications  as  well  as  their  periodic 
review  to  ensure  they  are  appropriate  for  changes  in 
realistic  operational  conditions.  It  emphasizes  that  the 
doctrine  and  tactics  must  "have  demonstrated  the  ca¬ 
pability  to  allow  commanders  and  operators  to  exer¬ 
cise  appropriate  levels  of  human  judgment  in  the  use 
of  force"  as  well  as  meet  all  legal  and  safety  rules.79 
Ideally,  the  development  and  promulgation  of  doc¬ 
trine  should  receive  high  priority  among  warfighters, 
but  this  is  not  always  the  case. 

In  his  book  Wired  for  War:  The  Robotics  Revolution 
and  Conflict  in  the  Twenty-first  Century,  author  P.  W. 
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Singer  devotes  a  chapter  to  examining  how  the  United 
States  might  fight  with  robots.  He  cited  a  survey  of  U.S. 
military  officers  that  "identified  developing  a  strategy 
and  doctrine  [for  using  robots  in  combat]  as  the  third 
least  important  aspect  to  figure  out."80  His  observation 
provides  some  insight  into  the  ad  hoc  applications  of 
AWS  in  combat  thus  far  despite  policy  that  dictates  a 
more  informed  approach. 

What  should  AWS  doctrine  address?  Perhaps  a 
good  starting  point  is  to  embrace  a  mission-focused 
autonomy  vision  crafted  by  the  ONR:  "Develop  au¬ 
tonomous  control  that  intelligently  understands  and 
reasons  about  its  environment  relative  to  its  objectives 
and  independently  takes  appropriate  actions."81  How¬ 
ever,  these  actions  in  a  mission  will  result  from  human 
direction  in  two  broad  teamwork  styles:  remote  pres¬ 
ence  or  taskable  agents.  As  its  name  implies,  remote 
presence  allows  humans  to  perceive  and  act  through 
unmanned  systems  at  a  distance,  while  taskable 
agents  allows  the  unmanned  system  to  autonomously 
complete  a  human-directed  mission  task.82 

Starting  with  the  foundation  of  remote  presence 
and  taskable  agents,  there  are  numerous  doctrinal  nu¬ 
ances  that  must  be  addressed  concerning  how  these 
principles  are  applied  in  operations  of  increasing 
complexity.  What  is  the  primary  operational  entity  for 
a  given  mission?  Are  humans  the  supported  force  ele¬ 
ment,  or  are  they  supporting  the  unmanned  system? 83 
Do  these  roles  change  at  different  phases  in  the  mis¬ 
sion?  How  autonomous  are  AWS  allowed  to  oper¬ 
ate  when  factoring  not  only  inherent  capability,  but 
also  commander's  intent,  laws  of  war,  and  rules  of 
engagement? 

Probably  the  best  attempt  at  addressing  such  AWS 
doctrine  issues  thus  far  occurred  at  Fort  Bliss,  Texas, 
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as  part  of  the  operational  assessment  of  fledging  FCS 
operations.84  But  these  experiments  were  limited  to 
few  soldiers  and  unmanned  systems.  However,  as  we 
move  toward  more  of  a  MUM-T  structure,  it  may  be¬ 
come  common  or  a  single  soldier  or  system  to  operate 
multiple  AWS  simultaneously;  an  extreme  application 
of  this  would  be  swarming. 

The  DSB  report  on  autonomy  discussed  two  possi¬ 
ble  types  of  swarming  coordination:  cooperation  and 
collaboration.  Cooperation  may  emulate  swarming 
behavior  seen  in  nature  among  large  numbers  of  ani¬ 
mals  executing  simple  tasks  based  on  instinct,  such  as 
ants  foraging  for  food.  Cognitive  and  communication 
abilities  for  cooperation  are  basic  and  may  be  reduced 
to  a  small  number  of  stimulus-response  commands. 
Collaboration  is  a  more  intentional  form  of  teamwork 
that  requires  more  sophisticated  sensing  and  commu¬ 
nication  as  well  as  cognitive  understanding  of  other 
members  of  the  swarm.85  Using  AWS  in  swarms  may 
provide  greatly  enhanced  mass  and  maneuver  capabil¬ 
ities  as  well  as  allow  for  the  use  of  lower  cost  systems 
in  large  enough  numbers  to  account  for  attrition  and 
possibly  overwhelm  any  traditional  defenses.  How¬ 
ever,  to  be  successful  it  is  likely  that  such  formations 
will  communicate  and  react  at  speeds  and  complexity 
beyond  human  comprehension;  this  introduces  a  limi¬ 
tation  that  doctrine  should  address. 86 

Doctrine  must  also  address  how  U.S.  human/ 
AWS  teams  interact  with  external  parties.  Such  inter¬ 
action  may  involve  coalition  teaming  efforts,  adver¬ 
sary  countermeasures,  and  incidental  contact  with 
innocent  noncombatants.87  To  better  understand 
the  possible  implications  of  such  interaction,  let  us 
now  examine  AWS  applications  in  the  international 
environment. 
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International  Efforts. 


The  global  proliferation  of  UAVs  as  depicted  in 
Figure  2  and  discussed  earlier  applied  to  all  forms  of 
aviation.  The  Bard  Center  for  Study  of  the  Drone  re¬ 
ports  that  by  2013,  87  countries  are  either  developing 
or  acquiring  military  drones.88  Filtering  this  number 
down,  a  2014  RAND  Corporation  report  found  23 
countries  are  potentially  developing  armed  UAVs, 
of  which  eight  countries  are  working  on  systems  that 
would  meet  Category  I  criteria  of  the  Missile  Technol¬ 
ogy  Control  Regime  (MTCR)  — a  range  of  at  least  300 
kilometers  with  a  payload  of  at  least  500  kilograms.  Of 
these  eight  countries,  three  are  MTCR  members  (Rus¬ 
sia,  Turkey,  and  the  United  States)  and  five  are  not 
members  (China,  India,  Iran,  Taiwan,  and  the  United 
Arab  Emirates). 

Of  note  is  that  Israel,  the  largest  exporter  of  UAVs 
(to  over  42  countries),  is  not  an  MTCR  member.  Israel 
is  assessed  as  developing  MTCR  Category  II  UAVs  — 
the  same  range  criteria  as  Category  I,  but  with  less 
payload.89  Israel  also  exports  ground  systems  like  the 
Guardium  unmanned  ground  vehicle  (UGV),  a  semi- 
autonomous  system  designed  primarily  for  routine 
patrols,  but  also  able  "to  autonomously  react  to  un¬ 
scheduled  events,  in  line  with  a  set  of  guidelines  spe¬ 
cifically  programmed  for  the  site  characteristics  and 
security  doctrine."90  China  is  also  an  exporter  of  AWS, 
according  to  the  DSB  which  noted  that  after  their  ini¬ 
tial  marketing  at  the  Zhuhai  air  show  in  2007,  "now 
every  major  manufacturer  for  the  Chinese  military  has 
a  research  center  devoted  to  unmanned  systems."91 
Other  countries  of  interest  for  exporting  AWS  include 
Russia  and  Iran.92 
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Given  the  crowded  international  environment  for 
military  unmanned  systems,  how  does  the  United 
States  extend  its  force  integration  of  AWS  to  include 
allies,  coalition  partners,  and  friends?  DoD  Directive 
3000.09  allows  for  the  "international  sales  or  transfers 
of  autonomous  and  semi-autonomous  weapon  sys¬ 
tems"  in  accordance  with  existing  U.S.  law.93  Chapter 
8  of  the  Integrated  Roadmap  identifies  three  primary 
methods  of  international  cooperation  for  unmanned 
systems:  international  agreements  for  joint  RDT&E; 
foreign  military  sales;  and  direct  commercial  sales.  It 
also  provides  details  regarding  the  rigorous  control 
processes  established  by  Congress  to  ensure  proper 
arms  export  control,  technology  transfer,  and  foreign 
disclosure  of  sensitive  information.94 

An  example  of  international  cooperation  is  the 
North  Atlantic  Treaty  Organization  (NATO)  Alliance 
Ground  Surveillance  (AGS),  a  system  that  uses  the 
U.S.  Global  Hawk  UAV  to  give  NATO  commanders 
comprehensive  ground  situational  awareness.  It  was 
used  successfully  to  help  protect  civilians  in  Libya 
during  NATO  operations  there  in  2011. 95  The  DoD  FY 
2016  budget  includes  $198  million  for  RDT&E  efforts 
for  the  AGS.96 

Individual  countries  have  benefited  from  U.S. 
partnerships  with  AWS  as  well.  High-demand  sys¬ 
tems  such  as  the  RQ-1  Predator  and  MQ-1  Reaper 
UAVs  built  by  General  Atomics  have  been  purchased 
by  many  countries,  including  NATO  allies  Canada, 
France,  Germany,  Italy,  Spain,  The  Netherlands,  and 
UK.97  The  Australian  Army  acknowledged  the  ben¬ 
efits  of  procuring  "capabilities  such  as  the  Switch¬ 
blade  system  (an  armed,  disposable  micro-UAV)"  to 
provide  "opportunities  for  the  [Australian]  Army  to 
enhance  the  individual  lethality  of  its  soldiers  in  an 
austere  budget  environment." 98 
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How  is  NATO  approaching  the  implementation  of 
AWS  into  future  operations?  A  series  of  workshops 
and  seminars  led  by  NATO  Headquarters  Supreme  Al¬ 
lied  Commander  Transformation  have  focused  on  the 
objective  "to  improve  awareness  and  understanding 
of  autonomous  systems,  promote  interoperability  and 
provide  guidance  for  the  development  of,  use  of,  and 
defence  against,  autonomous  systems."99  Their  work 
includes  recommendations  to  adopt  precise  language 
when  dealing  with  AWS,  specifically  the  introduction 
of  "autonomous  functioning"  as  reference  to  "the  abil¬ 
ity  of  a  system,  platform,  or  software,  to  complete  a 
task  without  human  intervention,  using  behaviours 
resulting  from  the  interaction  of  computer  program¬ 
ming  with  the  external  environment."100  Accordingly, 
the  report  recommendations  include  "rather  than 
emphasising  the  fact  that  a  system  employs  autono¬ 
mous  functions,  focus  should  be  placed  on  the  level 
of  human  control  and  accountability  and  the  type  of 
decision  being  autonomised."101  Such  clarification  ap¬ 
pears  to  be  in  concert  with  the  DoD  policy  guideline 
to  ensure  the  exercise  of  appropriate  levels  of  human 
judgment  when  using  AWS.102 

The  challenges  of  integrating  U.S.  military  un¬ 
manned  systems  into  operations  with  other  govern¬ 
mental  entities  are  not  limited  to  international  military 
forces.  There  are  also  issues  that  DoD  must  consider 
related  to  military  AWS  operations  in  the  continental 
United  States  and  bordering  countries.  For  example, 
a  2012  GAO  report  examined  the  use  of  UAVs  by  the 
Department  of  Homeland  Security  (DHS)  to  support 
national  security  operations  along  the  U.S.  southwest 
border.  The  Federal  Aviation  Administration  (FAA) 
granted  DHS  authority  to  fly  such  missions,  but  were 
concerned  about  safety  in  the  national  airspace,  since 
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the  UAVs  had  limited  capabilities  to  avoid  other 
aircraft.103 

The  January  2013  USD(AT&L)  report  to  Congress 
on  unmanned  systems  acknowledges  that  "current 
DoD  UAS  lack  the  same  capabilities  as  manned  air¬ 
craft  to  safely  and  efficiently  integrate  into  the  Na¬ 
tional  Airspace  System  (NAS)."104  The  DoD  UAS  Task 
Force  includes  an  integrated  product  team  dedicated 
to  address  this  problem: 

The  Airspace  Integration  IPT  (AI-IPT)  seeks  to  im¬ 
prove  airspace  access  for  UAS  operations  and  training 
requirements  in  support  of  homeland  defense,  home¬ 
land  security,  and  defense  support  of  civil  authorities. 
The  AI-IPT  reviews  and  assesses  operational  require¬ 
ments;  identifies  and  develops  acquisition  solutions; 
assists  in  the  development  of  UAS  technical  stan¬ 
dards;  and  recommend  straining  and  policy  changes 
necessary  to  integrate  UAS  into  necessary  classes  of 
airspace.105 

The  current  process  allows  the  FAA  to  issue  a  Cer¬ 
tificate  of  Waiver  or  Authorization  (CO A)  for  govern¬ 
ment  entities  that  desire  to  fly  a  UAS  in  civil  airspace 
for  purposes  such  as  law  enforcement,  firefighting, 
border  patrol,  disaster  relief,  search  and  rescue,  and 
military  training.  The  FAA  evaluates  the  COA  request 
for  safety  and,  if  approved,  defines  the  allowed  air¬ 
space  and  any  special  provisions.  The  numbers  of 
CO  As  granted  has  risen  from  146  in  2009  to  609  in 
2014. 106  To  further  complicate  the  situation,  as  of  May 
2014,  35  states  have  introduced  legislation  to  restrict 
the  use  of  drones  by  government;  12  state  legislatures 
have  actually  some  formal  measures.107  Returning  full 
circle  back  to  the  international  environment,  it  is  im¬ 
portant  to  note  that  the  European  Union  (EU)  is  also 
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considering  airspace  management  restrictions  for 
unmanned  air  systems.108  Also,  a  March  2015  GAO 
report  compares  the  evolution  of  airspace  restric¬ 
tions  for  UAS  operation  amongst  the  United  States, 
Australia,  Canada,  France,  and  UK.109 

Strategic  Geopolitical  Challenges. 

What  long-term  strategic  effects  on  American  na¬ 
tional  security  policy  are  likely  to  result  from  increased 
use  of  military  AWS?  When  examining  this  question, 
one  must  ask  if  the  AWS  are  simply  an  evolution  of 
warfare,  or  whether  they  may  represent  a  revolution 
in  military  affairs.110  The  2014  report  from  NATO  Al¬ 
lied  Command  Transformation  recommends  serious 
study  regarding  the  impact  of  autonomous  system  to 
the  character  of  war,  specifically  "the  potential  of  au¬ 
tonomous  technologies  to  impact  the  nature  and  con¬ 
duct  of  war."111  Consistent  with  this  theme,  the  report 
concludes  that  one  of  the  study's  most  important  find¬ 
ings  is  the  need  to  consider  the  proper  balance  of  hu¬ 
man  control  versus  machine  control.  Toward  this  end, 
the  report  recommends  this  be  done  at  policy  levels 
and  include  the  following  factors: 

•  The  type  of  decision  being  transferred  to  the 
machine; 

•  The  command  relationship  between  human 
and  machine; 

•  The  type  of  operating  environment; 

•  The  type  of  risk  incurred  if  the  machine  makes 
the  wrong  decision;  and, 

•  The  particular  military  benefit  of  autonomiza- 
tion  of  certain  functions  (e.g.  precision  perfor¬ 
mance,  faster  decisionmaking,  reduction  of  risk 
to  personnel).112 
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In  his  study  of  the  effectiveness  of  recent  armed 
drone  strikes,  author  James  Walsh  considers  the  long¬ 
term  implications  and  asks  "How  will  other  countries 
and  insurgent  organizations  respond  to  the  use  of 
drones  as  a  U.S.  tool  of  counterinsurgency?"113  Use  of 
drones  can  be  seductive,  since  such  operations  require 
fewer  "boots  on  the  ground"  and  thus  diminish  the 
risk  of  U.S.  casualties.  However,  a  2014  RAND  Corpo¬ 
ration  report  notes  that  this  reduced  fear  of  loss  may 
"change  the  calculus  for  the  employment  of  force"  to 
lower  the  threshold  of  intervention  and  warns  that 
"as  armed  UAVs  spread,  other  countries  may  be  more 
likely  to  intervene  in  similar  circumstances."114  A 
2014  study  of  future  warfare  by  the  Australian  Army 
echoed  the  need  for  carefully  examining  the  poten¬ 
tially  profound  changes  that  autonomous  technology 
may  have  in  future  wars.  The  report  singles  out  armed 
AWS,  stressing  that  "significant  questions  remain  con¬ 
cerning  the  ethics  and  legality  of  arming  autonomous 
platforms  and  empowering  these  systems  to  use  lethal 
force."115 

On  the  domestic  political  front,  Walsh  poses  a  ques¬ 
tion  that  would  please  Clausewitz:  "How  the  reliance 
on  drone  strikes  will  influence  perceptions  on  the  part 
of  the  American  public  of  the  acceptability  and  desir¬ 
ability  of  the  use  of  force."116  Certainly,  the  adminis¬ 
tration  of  President  Obama  has  greatly  escalated  the 
use  of  armed  drones  in  the  application  of  U.S.  force, 
including  use  "over  at  least  475  decisions  to  carry  out 
lethal  force  in  nations  beyond  the  ones  where  U.S. 
military  forces  are  deployed  on  the  ground."117  An  ap¬ 
parent  side  effect  to  this  increased  use  is  reduced  op¬ 
erational  and  political  costs  at  home,  a  trend  that  may 
make  the  decision  to  use  force  easier  in  the  future. 
Many  respected  government  experts  have  questioned 
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if  the  authorities  granted  by  such  measures  as  the  War 
Power  Act  have  been  interpreted  too  liberally  in  these 
operations.  To  address  such  concerns,  let  us  examine 
the  related  legal  issues. 

LEGAL  ISSUES  RELATED  TO  USE  OF 
AUTONOMOUS  SYSTEMS 

What  are  legally  acceptable  uses  of  autonomous 
weapon  systems  within  the  range  of  military  opera¬ 
tions?  To  examine  this,  we  will  first  look  at  existing 
international  legal  conventions  including  the  Law  of 
Armed  Conflict  (LOAC).  We  will  then  review  recent 
assessments  by  the  UN  LOAC.  Next,  we  focus  on  the 
concept  of  human  control  and  judgement  as  it  ap¬ 
plies  to  AWS.  Finally,  we  shall  review  some  current 
vignettes  as  well  as  consider  some  potential  trends  for 
the  future. 

International  Law  and  LOAC. 

Although  the  widespread  use  of  AWS  in  warfare 
is  still  in  its  infancy,  there  have  been  serious  concerns 
raised  regarding  the  legality  of  their  application.  What 
forms  the  legal  foundation  for  the  use  of  AWS  in  war¬ 
fare?  DoD  Directive  3000.09  explicitly  assigns  respon¬ 
sibility  to  the  General  Counsel  of  DoD  (GC,  DoD)  to 
"provide  for  guidance  in  and  coordination  of  legal 
reviews  of  weapon  systems  submitted  in  accordance 
with"  provisions  of  the  directive.118  This  includes  re¬ 
view  of  the  Law  of  War  (often  called  law  of  armed 
conflict,  or  LOAC)  per  the  provisions  of  DoD  Directive 
2311.01E,  DoD  Law  of  War  Program,  which: 

encompasses  all  international  law  for  the  conduct  of 

hostilities  binding  on  the  United  States  or  its  indi- 
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vidual  citizens,  including  treaties  and  international 
agreements  to  which  the  United  States  is  a  party,  and 
applicable  customary  international  law.119 

To  address  the  growing  controversy  regarding 
drone  attacks,  internationally  recognized  legal  expert 
Michael  Schmitt  published  a  detailed  legal  analysis 
that  examined  both  the  jus  as  bellum  (right  to  war) 
and  jus  in  hello  (international  humanitarian  law)  prin¬ 
ciples.  In  general,  he  concluded  "that  there  is  little 
reason  to  treat  drones  as  distinct  from  other  weapons 
systems  with  regard  to  the  legal  consequences  of  their 
employment."120  He  also  addressed  the  debate  over 
U.S.  drone  attacks  over  countries  whose  government 
did  not  grant  permission,  such  as  Pakistan.  Schmitt 
summarized  the  U.S.  justification  as  one  derived  from 
Article  51  of  the  UN  Charter  (inherent  right  of  self-de¬ 
fense),  stating  that  "the  victim  state  [the  U.S.]  may  act 
militarily  in  self-defense,  including  the  use  of  drones, 
to  put  an  end  to  the  unlawful  activities"  that  the  terri¬ 
torial  state  (Pakistan)  fails  to  stop.121  Schmitt's  analysis 
included  two  other  interesting  concepts  related  to  the 
use  of  drones.  First,  because  of  their  capabilities  for 
long  duration  loiter  and  precision  strike,  drones  may 
in  some  cases  be  the  most  legally  responsible  choice  of 
force  application.  Second,  drone  operators  located  in 
the  United  States  remain  legitimate  targets  of  enemy 
attack.122 

When  considering  the  use  of  armed  UAVs  to  tar¬ 
get  specific  individuals  or  groups,  the  Bard  College's 
Center  for  Study  of  the  Drone  assessment  resonated 
with  Schmitt's  arguments.  Specifically,  the  Center 
surmises  that: 

under  international  humanitarian  law,  the  United 
States  may  use  lethal  force  against  individuals  outside 
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of  an  active  war  zone,  but  only  if  these  individuals  are 
actively  involved  in  hostilities  that  pose  an  imminent 
threat  to  the  United  States  or  its  interests.123 

Recent  military  doctrine  of  UK  also  echoed  Schmitt's 
observation  that  AWS  may  in  some  cases  be  the  most 
responsible  choice.  They  offer  the  argument  regarding 
the  Counter-Rocket,  Artillery  and  Mortar  (C-RAM) 
employed  in  Afghanistan  that  "the  potential  damage 
caused  by  not  using  C-RAM  in  its  automatic  mode  jus¬ 
tifies  the  level  of  any  anticipated  collateral  damage."124 

Assessments  by  the  United  Nations. 

Thus  far,  the  discussion  has  focused  on  legal  is¬ 
sues  limited  to  AWS  in  the  air  domain;  a  more  general 
discussion  of  armed  AWS  in  all  domains  is  necessary 
to  properly  assess  the  legal  implications.  Toward  this 
objective,  the  UN  has  sponsored  two  meetings,  in  May 
2014  and  April  2015,  under  provisions  of  its  Conven¬ 
tion  on  Certain  Conventional  Weapons  (CCW)  fo¬ 
cused  on  "questions  related  to  emerging  technologies 
in  the  area  of  lethal  autonomous  weapons  [LAWS]."125 
The  May  2014  Meeting  of  Experts  on  LAWS  was  at¬ 
tended  by  representatives  from  87  countries  as  well 
as  over  25  nongovernmental  organizations  (NGOs) 
and  academe.  In  addition  to  discussing  legal  aspects 
of  LAWS,  the  agenda  included  sessions  on  technical 
issues,  ethics  and  sociology,  and  operational  and  mili¬ 
tary  aspects.  Since  this  was  the  inaugural  UN  meet¬ 
ing  on  LAWS,  it  generally  focused  on  identifying  the 
myriad  issues  and  working  toward  a  common  lexicon 
to  guide  future  discussion  toward  solutions. 

The  final  report  of  the  May  2014  meeting  indicated 
that  many  of  the  delegates  were  trying  to  consider  the 
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balance  between  "the  potential  for  rapid  technological 
developments  in  autonomous  weapons  to  radically 
transform  the  nature  of  warfare"  and  "the  necessity 
of  recognizing  the  significance  of  the  peaceful  uses 
of  autonomous  technologies  in  the  civilian  field."126 
While  there  was  no  agreement  on  any  significant  is¬ 
sue,  several  themes  emerged  in  the  technical  sessions, 
to  include  "the  notion  of  meaningful  human  control 
could  be  useful  to  address  the  question  of  autonomy" 
as  well  as  "the  concept  of  human  involvement  in  de¬ 
sign,  testing,  reviews,  training  and  use"  of  LAWS.127 
The  sessions  on  legal  aspects  revealed  diverse  views 
on  the  possibility  for  LAWS  to  comply  with  interna¬ 
tional  humanitarian  law  as  well  as  the  adequacy  of 
existing  international  law  to  address  potential  uses 
of  LAWS.  Another  provocative  issue  regarded  how 
responsibility  for  the  use  of  LAWS  would  be  deter¬ 
mined  and  whether  there  may  be  an  "accountability 
gap"  that  requires  an  examination  of  responsibilities 
for  not  only  LAWS  users,  but  also  for  those  involved 
with  the  design  and  manufacturing.128 

Based  on  the  progress  of  their  2014  meeting,  CCW 
leadership  organized  a  second  meeting  to  continue 
the  dialogue  on  LAWS.  In  preparation  for  this  meet¬ 
ing,  the  Center  for  a  New  American  Security  (CNAS) 
published  a  primer  for  delegates  of  the  second  meet¬ 
ing  based  in  part  on  key  issues  raised  in  the  first  meet¬ 
ing.  The  primer  encourages  that  "States  should  follow 
these  [LAWS]  discussions  with  a  more  focused  exami¬ 
nation  of  the  strategic  stability  issues  surrounding 
LAWS,  perhaps  in  the  form  of  a  working  group."129 
The  issues  listed  by  CNAS  included:  the  current  ex¬ 
istence  of  LAWS;  the  legality,  morality,  and  predict¬ 
ability  of  LAWS;  how  LAWS  affect  human  dignity 
and  ethics;  and  stabilizing  and  destabilizing  nature  of 
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LAWS  in  warfare.  They  also  highlighted  the  concept 
of  "meaning  human  control"  as  an  important  part  of 
addressing  the  other  challenge  areas  and  proposed  a 
working  definition: 

There  are  three  essential  components  of  meaningful 
human  control: 

1.  Human  operators  are  making  informed,  conscious 
decisions  about  the  use  of  weapons. 

2.  Human  operators  have  sufficient  information  to  en¬ 
sure  the  lawfulness  of  the  action  they  are  taking,  given 
what  they  know  about  the  target,  the  weapon,  and  the 
context  for  action. 

3.  The  weapon  is  designed  and  tested,  and  human  op¬ 
erators  are  properly  trained,  to  ensure  effective  con¬ 
trol  over  the  use  of  the  weapon. 

These  standards  help  ensure  accountability,  moral 
responsibility,  and  the  ability  to  safely  control  the 
weapon.130 

The  second  Meeting  of  Experts  on  LAWS  was  held 
April  13-17,  2015,  and  included  delegates  from  over 
90  countries  as  well  as  from  the  EU  and  15  NGOs. 
Observers  at  the  proceedings  noted  that  there  was 
positive  discussion,  but  still  no  true  consensus  on 
any  significant  issues.  At  the  most  fundamental  level, 
there  remains  some  disagreement  as  to  what  consti¬ 
tutes  LAWS,  especially  with  regard  to  existing  sys¬ 
tems,  such  as  armed  drones.  There  was  also  discus¬ 
sion  of  the  dilemma  surrounding  the  dual-use  nature 
of  autonomy  and  the  concern  that  a  universal  ban  on 
LAWS-related  technologies  may  inadvertently  outlaw 
some  beneficial  applications  as  well.131 

While  the  formal  report  on  the  2015  meeting  is 
still  being  written  and  reviewed  within  UN  chan¬ 
nels,  many  of  the  written  statements  of  contributing 
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countries  have  posted  on  the  CCW  website.  As  one 
might  expect,  both  meetings  included  representatives 
from  the  five  permanent  members  of  the  UN  Security 
Council  (China,  France,  Russian,  UK,  and  the  United 
States),  any  of  which  could  wield  veto  power  over 
enactment  of  any  eventual  LAWS-related  security 
resolutions.  The  opening  statement  by  the  UK  clearly 
established  their  viewpoint:  "From  our  perspective, 
to  discuss  LAWS  is  to  discuss  means  and  methods  of 
warfare.  As  such,  international  humanitarian  law  pro¬ 
vides  the  appropriate  paradigm  for  discussion."132 

The  U.S.  delegation  clarified  the  U.S.  position  that 
the  conference  discussion  should  only  involve  future 
systems  and  "emerging  technologies,"  and  that  for  the 
purposes  of  the  meeting,  "we  are  not  referring  to  re¬ 
motely  piloted  aircraft,  which  as  their  name  implies 
are  not  autonomous  weapons,  or  other  existing  weap¬ 
ons  systems."133  The  statement  included  "The  United 
States  has  a  process  in  place,  applicable  to  all  weapon 
systems,  which  is  designed  to  ensure  weapons  oper¬ 
ate  safely,  reliably  and  are  understood  by  their  human 
operators"134  and  further  noted  that  DoD  Directive 
3000.09  imposes  additional  scrutiny  for  any  poten¬ 
tial  LAWS,  but  "neither  encourages  nor  prohibits  the 
development  of  such  future  systems."135 

Human  Control  and  Judgment. 

In  his  closing  remarks  at  the  2015  LAWS  meetings, 
Paul  Scharre  of  CNAS  commented  "There  seems  to  be 
an  emerging  consensus  that  human  control  and  judg¬ 
ment  is  needed.  And  most  seem  to  agree  that  there 
should  be  a  necessary  quality  to  that  control,  just  as 
there  is  with  the  use  of  weapons  today."136  Although 
one  might  dismiss  this  as  a  self-fulfilling  prophecy 
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facilitated  by  the  CNAS  primer,  it  is  clear  the  issue  of 
human  control  and  judgment  is  an  essential  element 
to  include  in  any  deliberations  of  AWS.  For  example, 
the  International  Committee  of  the  Red  Cross  (ICRC) 
held  an  Expert  Meeting  of  Autonomous  Weapon  Sys¬ 
tems  in  March  2014,  the  results  of  which  informed 
the  LAWS  meetings.137  The  ICRC  meeting  included 
representatives  from  21  countries  and  13  NGOs  and 
their  findings  included  "recognition  of  the  importance 
of  maintaining  human  control  over  selecting  and  at¬ 
tacking  targets,  although  there  is  less  clarity  on  what 
would  constitute  'meaningful  human  control'. "13s  To 
help  explore  this  conceptual  ambiguity,  presentations 
given  at  the  meeting  included  topics  on  Human  Su¬ 
pervisory  Control;  Distinction,  Proportionality,  and 
Precaution;  and  Accountability  and  Responsibility. 
Pointing  to  ethical  considerations  beyond  legal  fac¬ 
tors,  the  report  notes  "it  is  argued  that  the  manner 
in  which  people  are  killed  matters,  even  if  they  are 
lawful  targets."139 

The  2014  NATO  Allied  Command  Transformation 
report  also  addresses  the  importance  of  considering 
human  control,  and  notes  in  general  terms  "the  neces¬ 
sary  level  of  human  control  depends  on  the  particular 
situation,  applicable  legal  constraints,  and  the  level  of 
tolerable  risk."140  For  unarmed  AWS,  risk  assessment 
may  be  based  on  the  potential  for  collateral  damage 
based  on  the  size  of  the  system  and  the  possible  kinet¬ 
ic  energy  if  it  crashed.141  The  report  further  notes  the 
more  severe  restriction  case  of  LAWS  application  and 
concludes,  "The  idea  that  autonomous  systems  could 
be  autonomously  deciding  on  the  use  of  lethal  force 
against  humans,  is  perceived  by  some  as  being  incom¬ 
patible  with  the  dictates  of  public  conscience."142 
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Toward  the  Future. 


Despite  the  insistence  of  the  U.S.  delegation  to  the 
recent  UN  meeting  that  armed  drones  are  not  associ¬ 
ated  with  LAWS  issues,  it  may  be  prudent  to  consider 
the  opinions  of  the  rest  of  the  world.  The  first  openly 
acknowledged  successful  use  of  an  armed  U.S.  UAV 
was  a  Predator  in  Afghanistan  on  October  7,  2001. 143 
The  legal  justification  for  the  attack  was  based  in  large 
part  on  the  "Authorization  for  Use  of  Military  Force" 
passed  as  a  joint  resolution  by  Congress  following  the 
9/11  attacks.144  A  report  by  the  RAND  Corporation 
found  that  since  that  first  use,  there  have  been  eight 
other  conflicts  in  which  the  United  States  used  armed 
UAVs.  Per  the  report's  assessment  of  these  conflicts, 
three  did  not  occur  in  a  recognized  war  zone  (Paki¬ 
stan,  Yemen,  and  Somalia),  and  three  did  not  have 
clear  congressional  authorization  (Yemen,  Libya,  and 
Somalia).  Also,  the  report  posited  that: 

the  greatest  concerns  about  U.S.  use  of  armed  UAVs 
appear  to  arise  from  operations  outside  active  war 
zones,  less-transparent  operations,  lack  of  clarity 
about  congressional  authorizations,  and  targeting  of 
those  not  clearly  identified  as  combatants  or  al  Qaeda 
leaders.145 

In  a  Parameters  article,  W.  Andrew  Terrill  explored 
the  use  of  drones  over  Yemen  in  further  detail  and 
noted  that: 

despite  their  successes,  the  use  of  US  drones  is  deeply 
unpopular  with  many  Yemenis,  and  anger  over  their 
employment  is  one  of  the  primary  drawbacks  to  us¬ 
ing  these  systems.  One  of  the  most  important  rea¬ 
sons  for  Yemeni  anger  is  a  concern  about  national 
sovereignty.146 
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Terrill  noted  the  resulting  reluctance  of  (former) 
President  Ali  Abdullah  Saleh  to  publicly  acknowledge 
Yemen  cooperation  with  the  United  States  for  drone 
strikes.  Even  with  the  2012  election  of  President  Abed 
Rabbu  Hadi  and  a  more  openly  cooperative  relation¬ 
ship  with  the  United  States,  Terrill  recommended  that 
"US  leadership  correspondingly  needs  to  avoid  view¬ 
ing  its  drone  program  as  a  panacea  for  Yemen's  terror¬ 
ism  and  insurgency  problems."147  In  a  rather  prophetic 
conclusion,  Terrill  asserted: 

In  sum,  drones  are  on  probation  with  the  Yemeni  pub¬ 
lic,  and  even  a  friendly  Yemeni  president  can  still  be 
pressured  to  disallow  drone  strikes.  Drones  can  help 
manage  instability  but  they  cannot,  by  themselves, 
create  stability  in  Yemen.148 

Since  the  publication  of  the  Parameters  article  in  2013, 
the  government  of  Yemen  has  devolved  into  civil  war 
in  March  2015  with  rebel  forces  backed  by  Iran  forc¬ 
ing  President  Hadi  to  flee  to  Saudi  Arabia.149  Granted, 
the  dynamics  in  Yemen  go  far  beyond  a  simple  cause- 
and-effect  equation  with  armed  UAVs,  and  the  even¬ 
tual  benefit,  or  detriment,  of  their  use  requires  further 
study. 

But  even  in  a  legitimate  conflict  with  the  support 
of  local  government,  commanders  and  operators  of 
AWS  need  to  minimize  collateral  damage.  The  2014 
Australian  Army  Future  Land  Warfare  Report  notes 
that  "the  increasing  difficulty  of  discriminating  be¬ 
tween  combatants  and  noncombatants  is  likely  to 
require  more  extensive  targeting  preparation  and  an 
increased  need  for  target  audience  analysis."150  Thus, 
governments  need  to  be  attuned  to  public  awareness 
of  battlespace  operations  and  any  implicit  promise  of 
war  without  casualties.  Enabled  by  ubiquitous  social 
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media,  the  report  suggests  "the  gap  between  percep¬ 
tions  of  bloodless  precision  and  the  reality  of  close 
combat  will  be  difficult  to  bridge."151 

It  will  be  even  tougher  to  develop  appropriate  ar¬ 
tificial  intelligence  for  AWS  to  discriminate  targets  for 
application  of  lethal  force.  The  challenge  is  so  great 
and  the  implications  so  severe  that  the  European  Par¬ 
liament  has  called  on  EU  member  states  to  consider 
to  "ban  the  development,  production,  and  use  of  fully 
autonomous  weapons  which  enable  strikes  to  be  car¬ 
ried  out  without  human  intervention."152  But  even  if 
these  countries  and  the  United  States  impose  such 
restrictions,  potential  adversaries  may  not  follow  con¬ 
straints.  The  2014  Australian  Army  report  sums  this 
up  well:  "Legal,  moral  or  ethical  constraints,  which 
are  deemed  to  uphold  the  legitimacy  and  legality  of 
Western  military  operations,  are  unlikely  to  restrict 
the  actions  of  potential  adversaries."153 

The  path  forward  for  the  United  States  will  con¬ 
tinue  to  be  watched  closely  by  the  rest  of  the  world. 
As  a  2014  report  for  the  Council  for  Foreign  Relations 
cautions: 

Given  that  the  United  States  is  the  lead  actor  and  ex¬ 
emplar  of  drone  use,  its  interpretation  of  international 
law,  public  articulation  of  its  position,  and  future  be¬ 
havior  will  set  a  precedent  on  which  other  countries 
are  like  to  base  their  own  behavior.154 

These  strategic  decisions  will  have  implications  for 
the  future  of  all  AWS,  not  just  UAVs.  Rather  than  fo¬ 
cusing  exclusively  on  the  myriad  tactic  legal  details 
involved  with  AWS  and  LAWS,  perhaps  the  United 
States  should  also  promulgate  a  broader  vision  of 
ethical  behavior  that  can  help  promote  a  culture  of 
responsible  use  of  force.  Rebecca  Crootof  of  Yale  Law 
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School  studied  international  law  for  autonomous 
weapons  in  the  context  of  LOAC  and  beyond.  Her  rec¬ 
ommendation  is  straightforward:  "Given  that  we  do 
not  yet  fully  understand  the  benefits  and  risks  posed 
by  autonomous  weapons  systems,  developing  flexible 
codes  of  conduct  may  be  preferable  to  negotiating  a 
treaty."155  Indeed,  developing  such  codes  of  conduct 
requires  a  dialogue  that  extends  beyond  the  low  bar  of 
the  interpretation  of  laws  and  considers  the  relevant 
ethical  issues. 

ETHICAL  ISSUES  RELATED  TO  USE  OF 
AUTONOMOUS  SYSTEMS 

What  should  be  the  ethically  acceptable  and  advis¬ 
able  uses  of  autonomous  weapon  systems  within  the 
range  of  military  operations?  What  moral  principles 
should  form  the  foundation  of  AWS  development  and 
operation?  We  explore  these  questions  by  first  review¬ 
ing  the  ongoing  work  for  developing  ethical  frame¬ 
works  for  AWS.  We  then  consider  the  varying  cul¬ 
tural  views  that  AWS  applications  may  evoke.  Next, 
we  analyze  the  potential  reduction  or  proliferation  in 
warfare  that  widespread  use  of  AWS  may  introduce. 
Finally,  we  look  toward  the  future  and  contemplate 
potential  long-term  effects  on  national  security. 

Ethical  Frameworks. 

The  report  of  the  2014  LAWS  Expert  Meeting  in¬ 
cluded  a  section  that  summarized  a  session  held  to 
examine  ethical  and  sociological  aspects.  Several  note¬ 
worthy  themes  were  identified  for  future  discourse 
centered  on  the  inherent  deficiencies  of  inculcating 
ethics  into  machines.  Many  agreed  that  "the  possi- 
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bility  for  a  robotic  system  to  acquire  capabilities  of 
'moral  reasoning'  and  'judgment'  was  highly  ques¬ 
tionable."156  Some  of  the  perceived  difficulties  in  this 
endeavor  would  include  the  pragmatic  task  of  writ¬ 
ing  software  with  appropriate  values  and  ethics  that 
could  be  exercised  in  complex  and  dynamic  environ¬ 
ments.  The  report  also  questioned  how  autonomous 
systems  might  respond  to  moral  dilemmas.  Finally, 
even  if  these  challenges  were  surmounted,  could 
humans  fully  trust  such  systems  and  would  they  be 
acceptable  to  society  writ  large? 

Internationally  recognized  organizations  that  have 
dedicated  efforts  to  examine  the  ethics  involved  with 
LAWS  include  the  Human  Rights  Watch  and  the  In¬ 
ternational  Human  Rights  Clinic  (IHRC),  part  of  the 
Human  Rights  Program  at  Harvard  Law  School.  Since 
November  2012,  these  groups  joined  to  publish  three 
reports  on  the  dangers  posed  by  "killer  robots,"  which 
they  define  as  fully  autonomous  weapons  that  "pos¬ 
sess  the  ability  to  select  and  engage  their  targets  with¬ 
out  meaningful  human  control."157  Their  first  report, 
Losing  Humanity:  The  Case  against  Killer  Robots  (No¬ 
vember  2012)  builds  a  case  for  the  possibility  of  killer 
robots  becoming  reality  within  20  to  30  years  as  well 
as  an  urgency  for  their  recommendation  for  appropri¬ 
ate  national  and  international  measures  "to  prohibit 
the  development,  production,  and  use  of  fully  au¬ 
tonomous  weapons."158  The  second  report,  Shaking  the 
Foundations,  The  Human  Rights  Implications  of  Killer  Ro¬ 
bots  (May  2014)  emphasizes  the  ethical  concepts  of  hu¬ 
man  dignity  and  right  to  life.  Its  conclusion  includes: 

Finally,  as  machines,  fully  autonomous  weapons  could 

not  comprehend  or  respect  the  inherent  dignity  of  hu¬ 
man  beings.  The  inability  to  uphold  this  underlying 
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principle  of  human  rights  raises  serious  moral  ques¬ 
tions  about  the  prospect  of  allowing  a  robot  to  take  a 
human  life.159 

The  third  and  most  recent  report,  Mind  the  Gap:  The 
Lack  of  Accountability  for  Killer  Robots  (April  2015)  fo¬ 
cuses  on  a  very  direct  premise:  "Because  these  robots 
would  be  designed  to  kill,  someone  should  be  held 
legally  and  morally  accountable  for  unlawful  killings 
and  other  harms  the  weapons  cause."160  The  report  ar¬ 
gues  that  existing  immunity  granted  to  military  oper¬ 
ators  and  contractors  represent  gaps  in  accountability 
and  that  such  failure  to  uphold  responsibility  "add  to 
the  moral,  legal,  and  technological  case  against  fully 
autonomous  weapons  and  bolster  the  call  for  a  ban  on 
their  development,  production,  and  use."161 

Two  of  the  positions  endorsed  by  Human  Rights 
Watch  and  IHRC  complement  earlier  discussions  in 
this  Paper.  First  is  their  participation  in  the  UN  LAWS 
Meetings  of  Experts  and  specifically  their  recognition 
of  the  role  of  human  control.  In  their  opening  state¬ 
ment  at  the  2014  meeting,  their  representative,  Steve 
Goose,  noted  "The  key  to  success  this  week  will  be 
the  beginning  of  the  emergence  of  a  consensus  that 
there  should  always  be  meaningful  human  control  of 
the  targeting  and  kill  decisions  in  any  individual  at¬ 
tack  on  other  humans."162  Second  is  a  recommenda¬ 
tion  from  the  first  killer  robot  report  that  proposes  a 
worthwhile  initiative  for  those  involved  in  the  devel¬ 
opment  of  AWS: 

To  Roboticists  and  Others  Involved  in  the  Develop¬ 
ment  of  Robotic  Weapons 

•  Establish  a  professional  code  of  conduct  governing 
the  research  and  development  of  autonomous  ro¬ 
botic  weapons,  especially  those  capable  of  becoming 
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fully  autonomous,  in  order  to  ensure  that  legal  and 
ethical  concerns  about  their  use  in  armed  conflict  are 
adequately  considered  at  all  stages  of  technological 
development.163 

The  March  2014  ICRC  Expert  Meeting  Report  in¬ 
cluded  summaries  of  three  speakers  focused  specifi¬ 
cally  on  ethical  issues  surrounding  AWS.  Professor 
Ronald  Arkin  of  the  Georgia  Institute  of  Technology 
addressed  the  concept  of  ethical  restraint  of  LAWS.  He 
introduced  several  root  causes  of  war  crimes  attribut¬ 
able  to  human  motives  (such  as  revenge,  dehuman¬ 
ization,  immaturity,  frustration,  and  pleasure)  that 
could  be  improved  by  the  use  of  automated  systems. 
However,  he  admitted  that  autonomous  measures 
may  also  have  negative  impacts  on  squad  cohesion  as 
well  as  relations  with  local  populations.  In  the  end,  he 
argued  that  noncombatants  could  benefit  "though  the 
judicious  deployment  of  these  robotic  systems,  if  done 
carefully  and  thoughtfully,  particularly  in  those  com¬ 
bat  situations  where  fighters  have  a  greater  tendency 
or  opportunity  to  stray  outside  IHL."164 

Dr.  Peter  Asaro  of  The  New  School,  USA,  provided 
an  excellent  overview  of  seminal  works  of  ethics  and 
philosophy  that  inform  the  dialogue  on  AWS.  He  not¬ 
ed  the  foundations  set  forth  by  the  Martens  Clause  to 
evaluate  not  only  a  new  weapon  system's  compliance 
with  IHL  but  also  its  adherence  to  matters  of  human¬ 
ity  and  public  conscience.  He  also  explained  some  of 
the  moral  frameworks  that  define  this,  such  as  the  UN 
Declaration  of  Human  Rights  (1948)  as  well  as  Western 
philosophical  tradition,  such  as  rights-based  (Kantian) 
theories  and  virtue  ethics.  Based  on  this,  he  posited 
that  "in  giving  over  the  responsibility  to  make  target¬ 
ing  decisions  to  machines,  we  fundamentally  change 
the  nature  of  the  moral  considerations  involved  in 
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the  use  of  violent  force."165  He  concluded  by  ponder¬ 
ing  the  implications  of  allowing  AWS  to  take  human 
lives,  asking  "As  we  give  over  the  decisions  of  life  and 
death  to  technological  systems,  are  we  diminishing 
the  value  of  human  life?"166 

Finally,  Dr.  Peter  Lee  of  the  University  of  Ports¬ 
mouth  (UK)  posited  that  "such  a  thing  as  a  fully  auton¬ 
omous,  cognisant,  self-reasoning  weapon  system  does 
not  exist  [yet]"  and  thus  the  current  dialogue  about 
future  AWS  "is  necessarily  shaped  by  two  things: 
perceptions  of  the  nearest  equivalents  (drones,  cur¬ 
rently  remotely  piloted),  and  the  influence  of  science 
fiction  and  the  Hollywood  effect"  and  therefore  "any 
ethical  analysis  is  subject  to  contestation  and  lacking 
demonstrable  'facts'."167  He  also  outlined  "the  moral 
calculus  of  oversight  and  accountability"  by  compar¬ 
ing  the  evolution  of  the  "kill  chain"  from  World  War 
II  bombings,  to  Reaper  armed  drones,  and  then  to 
possible  AWS  operations.  He  concluded  by  arguing 
that  the  LAWS  debate  will  need  to  continue  for  years 
since  "ethical  assessments  of  autonomous  weapons 
are  currently  as  limited  as  the  technological,  military 
and  political  assumptions  they  are  based  upon."168  Dr. 
Lee  noted  the  potential  impact  that  popular  science 
fiction  may  have  on  perceptions  surrounding  AWS 
ethical  issues;  what  other  cultural  influences  impact 
this  discourse? 

Cultural  Views. 

Greg  Kennedy  in  a  2013  Parameters  article  exam¬ 
ined  some  of  the  cultural  views  of  other  countries  re¬ 
garding  war  by  proxy.  He  observes  that,  even  though 
U.S.  drone  attacks  may  have  military  success,  they  may 
also  increase  animosity  among  the  population  that 
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actually  can  increase  attacks  on  a  besieged  government 
that  the  United  States  is  attempting  to  bolster.  Even 
though  much  of  the  resentment  may  be  anti-American, 
unfortunately,  "feelings  of  hostility  are  often  visited 
on  the  most  immediate  structures  of  authority  [of  the 
local  government]."169  More  troubling  are  the  broader 
implications  "regarding  the  legality,  ethicality,  and 
operational  legitimacy  of  those  [U.S.]  acts  to  deter  op¬ 
ponents."170  Kennedy  concludes  that  evolution  from 
limited  covert  operations  to  widespread  use  of  armed 
drones  exacerbates  "the  apparent  gap  between  stated 
core  policies  and  values  and  the  ability  to  practice  tar¬ 
geted  killings  appears  to  be  a  starkly  hypocritical  and 
deceitful  position  internationally."171 

Two  additional  articles  from  the  same  issue  of  Pa¬ 
rameters  delve  into  some  of  the  nuances  of  Kennedy's 
article.  Jacqueline  L.  Hazelton  examined  how  the  use 
of  drone  strikes  compare  to  other  tools  of  the  state. 
She  argues  that  strikes  against  assets  that  are  planning 
an  imminent  attack  on  the  U.S.  homeland  present  dif¬ 
ferent  moral  questions  than  strikes  targeting  individu¬ 
als  that  may  be  acting  suspiciously.  Thus,  she  posits 
that  leaders  must  consider  the  full  political  context 
when  pondering  the  use  of  drone  strikes,  including 
"for  example,  theaters  where  the  United  States  is  at 
war,  theaters  in  which  it  is  not,  theaters  in  which  the 
United  States  has  national  or  international  permission 
to  strike,  theaters  in  which  it  does  not,  and  so  on."172 
In  his  article,  Alan  W.  Dowd  contends  further  that  the 
current  context  for  U.S.  decisionmakers  should  take 
into  account  how  "the  brewing  international  backlash 
against  the  drone  war  reminds  us  that  means  and 
methods  matter  as  much  as  ends."173 

An  example  illustrating  Dowd's  observation  re¬ 
garding  the  means  and  methods  can  be  found  in 
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French  author  Gregoire  Chamayou's  book,  Theorie 
du  drone  ( Drone  Theory)  in  the  chapter  titled  "Ethos  et 
psyche"  ("Ethos  and  psyche").  There  he  mulls  over 
the  unit  morale  patch  used  by  some  U.S.  Reaper  UAV 
operators  that  depicts  an  image  of  the  Grim  Reaper 
holding  a  bloody  scythe  under  a  heading  "USAF 
MQ-9  Reaper"  and  a  tab  with  the  motto  "That  Others 
May  Die."174  While  the  message  behind  the  image  may 
not  win  favor  amongst  international  audiences,  they 
may  be  even  more  dismayed  to  find  out  that  the  patch 
is  a  lampoon  of  the  U.S.  Air  Force  Pararescue  force 
which  uses  a  patch  of  the  same  design,  but  with  an 
angel  instead  of  the  Grim  Reaper  and  the  motto,  "That 
Others  May  Live."175 

The  2014  NATO  Allied  Command  Transformation 
report  exploring  policy  guidance  for  use  of  autonomy 
also  addressed  ethical  issues  and  recommended  trans¬ 
parency  for  policymakers  as  they  consider  potential 
benefits  and  concerns  related  to  AWS  technology. 
However,  the  scope  of  their  recommendation  went  be¬ 
yond  those  of  LAWS  to  include  the  review  of  "levels  of 
responsibility  for  the  intended  and  unintended  conse¬ 
quences  of  tasks  performed  by  autonomous  systems," 
and  it  admonishes  that  considerations  "should  not 
neglect  nonlethal  tasks  performed  autonomously."176 

Warfare:  Reduction  or  Proliferation? 

If  AWS  use  becomes  widespread,  how  will  it  affect 
the  frequency  and  intensity  of  conflict  and  the  applica¬ 
tion  of  force?  Despite  the  anticipated  improvements  in 
lethality  and  precision  in  future  AWS  that  may  better 
deter  aggression,  many  argue  that  the  threshold  for 
use  of  force  may  actually  be  lowered.  Thomas  Cowan 
explored  the  impact  of  robots  on  warfare  using  the 
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Clausewitz  model  of  war  — the  trinity  of  primordial 
violence  (depicted  as  the  people),  chance  (depicted 
as  the  military),  and  reason  (depicted  as  the  govern¬ 
ment)  —  as  the  lens  for  his  analysis.  He  posited  that  the 
use  of  "robots  will  significantly  increase  the  potential 
for  future  conflicts"  for  three  primary  reasons.  First, 
there  will  be  fewer  people  directly  involved  as  they 
are  replaced  by  machines;  second,  friction  will  be  re¬ 
duced  as  the  robots  do  not  suffer  from  human  physi¬ 
cal  or  mental  frailties;  and  third,  the  government  may 
operate  with  less  constraint  since  the  probability  of 
creating  friendly  casualties  is  reduced.177 

In  a  2002  Parameters  article,  D.  Keith  Shurtleff  ex¬ 
pounds  on  the  notion  of  how  increasing  technology 
may  diminish  warfighters'  humanity.  He  examines  the 
concept  of  disengagement,  often  facilitated  through 
video  recordings  such  as  "the  luckiest  man  in  Iraq"  — 
a  man  in  a  video  that  drove  a  truck  across  a  bridge  just 
before  a  precision  bomb  destroyed  the  bridge.  General 
Norman  Schwarzkopf  used  this  moniker  as  a  punch¬ 
line  during  an  Operation  DESERT  STORM  press  con¬ 
ference  as  many  others  have  done  in  a  similar  manner 
since.  Shurtleff  sums  up  the  dilemma  as: 

what  we  must  deal  with  now  is  how  to  accept,  even 
embrace,  technologies  that  make  war  safer,  and  yet 
somehow  counter  the  trend  that  such  technologies 
have  to  disengage  us,  to  make  war  more  acceptable  or 
potentially  more  ubiquitous.178 

An  interesting  phenomenon  that  runs  counter  to 
Shurtleff's  thesis  that  robots  may  reduce  our  basic 
humanity  involves  situations  where  soldiers  in  OEF / 
OIF  personified  "their"  robots  and  wanted  them  fixed 
because  of  their  "loyalty."  This  was  the  case  of  a  badly 
damaged  EOD  PackBot  named  "Scooby-Doo"  as  told 
in  Singer's  Wired  for  War.179 


56 


In  contrast,  in  his  2013  Parameters  article,  Dowd 
cautions  against  the  possible  development  of  a  "Jupi¬ 
ter  Complex"  by  national  leaders  — a  condition  coined 
during  World  War  II  to  describe  "the  notion  of  the 
Allies  as  righteous  gods  raining  retributive  thunder¬ 
bolts  on  their  wicked  enemies."180  The  promises  of  in¬ 
creased  combat  effectiveness  at  reduced  costs  can  be 
seductive. 

UCAVs  take  the  logic  of  the  Jupiter  Complex  to  its 
ultimate  conclusion— maximum  use  of  economic  and 
technological  resources  with  zero  manpower  losses 
and  zero  risks  —  all  buffered  by  the  virtual-reality  na¬ 
ture  of  the  delivery  system.181 

How  can  the  U.S.  leadership  steer  clear  of  such 
temptations? 

Toward  the  Future. 

A  report  on  AWS  under  International  Law  by  the 
Geneva  Academy  points  to  the  need  for  leaders  to  con¬ 
sider  the  balance  between  advantages  and  disadvan¬ 
tages  of  using  AWS  for  a  given  mission.  Possible  ad¬ 
vantage  include  "the  absence  of  emotions  such  as  fear, 
vengeance,  or  self-interest  [that]  may  lead  to  outcomes 
that  overall  are  less  harmful."  But  lack  of  feelings  may 
be  a  disadvantage  in  certain  situations  since  "autono¬ 
mous  weapon  systems  lack  positive  human  emotions, 
such  as  compassion  or  mercy."182  Dowd  notes  further 
that  "they  [UAS]  remove  the  unique  characteristics 
humans  bring  to  the  battlespace:  deliberation,  doubt, 
fear,  gut  instinct,  and  judgment." 183 

So  then,  how  should  leaders  resolve  the  tension 
between  these  perspectives?  Perhaps  a  good  starting 
point  is  a  group  of  questions  sent  to  representatives  of 
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the  2015  UN  LAWS  Meeting  as  "Food-for-thought"  to 
prepare  for  the  dialogue. 

Military  rationale  for  autonomous  functions  in  weapon 
systems. 

(a)  What  is  the  military  rationale  for  pursuing 
autonomy  in  specific  functions  of  weapons  systems? 
What  would  be  the  reasons  for  limiting  the  autono¬ 
mous  capabilities  of  weapon  systems? 

(b)  In  what  situations  are  distinctively  human 
traits,  such  as  fear,  hate,  sense  of  honor  and  dignity, 
compassion,  and  love,  desirable  in  combat?  In  what 
situations  do  machines  that  lack  emotions  offer  dis¬ 
tinct  advantages  over  human  combatants? 

(c)  International  humanitarian  law  indicates  how  a 
party  to  a  conflict  should  behave  in  relation  to  people 
at  its  mercy,  how  would  machines  comprehend  such 
situations? 

(d)  Given  the  volume  and  complexity  of  available 
information,  what  advantages  do  autonomous  pro¬ 
cesses  offer?  Do  these  make  a  machine  more,  or  less 
reliable  than  a  human  decisionmaker? 

(e)  Are  there  scenarios  in  which  autonomy  might 
help  to  protect  the  lives  of  civilians  and  combatants? 
For  example,  more  precise  targeting,  preventing  col¬ 
lateral  damage  and  lower  response  time  to  attacks? 

(f)  What  are  the  specific  opportunities  and  concerns 
for  developing  and  deploying  autonomous  weapons 
in  the  context  of  land,  sea,  and  air  operations?184 

Sources  of  recent  studies  and  literature  that  sup¬ 
port  thoughtful  deliberation  of  these  questions  can  be 
found  at  the  website  of  the  CNAS  Ethical  Autonomy 
Project  (this  includes  an  extensive  online  bibliography 
as  public  education  resource).185  With  such  deliberate 
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and  continuous  examination  of  the  ethical  issues  in 
addition  to  those  of  mission  effectiveness  and  legal¬ 
ity,  we  can  hopefully  escape  the  fate  offered  in  a  final 
warning  from  Dowd:  "It  would  be  ironic  if  the  promise 
of  risk-free  war  presented  by  drones  spawned  a  new 
era  of  danger  for  the  United  States  and  its  allies."186 

RECOMMENDATIONS 

This  Paper  explores  a  diverse  set  of  complex  issues 
related  to  the  developmental,  operational,  legal,  and 
ethical  aspects  of  AWS.  Many  of  these  issues  are  al¬ 
ready  on  the  agendas  of  competent  government,  non¬ 
government,  and  industry  organizations,  with  mixed 
progress  toward  full  understanding  and  resolution. 
The  section  offers  recommendations  to  facilitate  the 
best  evolutionary  path  for  the  future  of  some  of  the 
key  issues  affecting  the  use  of  AWS  to  enhance  U.S. 
national  security. 

Policy. 

Current  DoD  policy  appears  to  address  the  big 
issues  being  raised  internationally  in  such  venues  as 
those  sponsored  by  the  UN  and  the  ICRC.  The  next  it¬ 
eration  of  AWS  policy  in  DoD  Directive  3000.09  should 
expand  to  include  unarmed,  unmanned  platforms 
that  are  capable  of  causing  damage  to  individuals  or 
property  if  they  malfunction  or  if  their  command  and 
control  link  is  lost. 

DoD  should  abandon  discrete  definitions  that  pi¬ 
geonhole  systems  into  categories  of  autonomy  and 
instead  consider  the  recommendations  of  the  DSB 
regarding  the  characterization  of  autonomy.  Accord¬ 
ingly,  DoD  should  develop  a  framework  which  con- 
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siders  the  autonomous  functions  of  a  system  and  how 
they  may  change  during  the  course  of  a  given  mission. 
Thus,  it  is  more  important  to  clearly  identify  when 
autonomous  functioning  of  the  unmanned  system  is 
planned  to  occur  (intentionally  autonomous)  as  well 
as  how  and  when  it  can  occur  unintentionally  (either 
through  normal  malfunction,  interference,  or  attack  by 
adversary  on  the  system  or  its  links).  Existing  frame¬ 
works  to  consider  include  ones  proposed  by  CNAS 
(that  examines  human-machine  command  and  con¬ 
trol,  complexity  of  machine,  and  type  of  decision)187 
and  by  the  DSB  (that  examines  cognitive  echelon,  mis¬ 
sion  timelines,  and  human-machine  trade  space).188 

DoD  and  the  Army  need  to  evaluate  the  budget  for 
unmanned  ground  vehicle  development  to  assess  the 
realism  of  its  ability  to  achieve  any  serious  evolution 
of  capability  and  integration  into  the  ground  forces 
writ  large.  Any  such  review  should  attempt  to  justify 
resource  allocation  based  on  the  operational  merits  of 
UGV  systems  and  not  emphasize  dubious  claims  of 
cost  savings.189 

Maintaining  Humanity. 

DoD  policy  should  continue  to  require  measures 
that  allow  the  exercise  of  human  judgment  and  con¬ 
trol  in  AWS  operations.  Further,  this  requirement 
should  expand  to  include  greater  emphasis  for  com¬ 
manders  and  planners  to  consider  how  the  scope  of 
such  human  control  may  vary  during  a  given  mission 
as  well  as  the  potential  strategic  implications  of  any 
fully  autonomous  mission  segments.  Also,  the  AWS 
development  community  should  also  emphasize  how 
autonomy  and  human  control  may  vary  during  op¬ 
erations  and  design  fault-tolerate  systems  that  incor- 
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porate  "fail  safe"  modes  for  all  mission  segments,  not 
just  those  designated  to  be  intentionally  autonomous. 
The  development  community  should  also  establish 
and  promulgate  an  ethical  code  of  conduct  for  work 
on  AWS  similar  in  nature  to  that  proposed  by  Human 
Rights  Watch  and  IHRC. 

All  parties  involved  with  the  development  and  op¬ 
eration  of  AWS  need  to  ensure  the  human  dimension 
is  explicitly  emphasized  and  monitored  in  doctrine, 
organizations,  and  processes.  Leaders  and  planners 
need  to  be  vigilant  to  guard  against  any  form  of  the 
"Jupiter  Complex"  that  may  emerge  at  the  strategic 
level.  Developers  and  operators  need  to  guard  against 
any  unhealthy  disengagement  at  the  tactic  and  opera¬ 
tional  levels.190 

Strategic  Implications. 

The  mainstream  discussions  regarding  military 
unmanned  systems  and  AWS  are  dominated  by  issues 
related  to  the  use  of  armed  drones.  Popular  media  fo¬ 
cuses  on  the  exploits  of  UAVs,  from  possible  package 
delivery  vehicles  to  platforms  that  invade  privacy. 
Government  and  commercial  investors  are  pursuing 
the  development  and  integration  of  driverless  cars 
into  the  nation's  highway  system.  Many  of  these  ef¬ 
forts  involve  regulations  that  cut  across  government 
departments,  but  there  is  no  clear  consideration  for 
how  these  efforts  are  preceding  writ  large. 

The  long-term  implications  for  current  tactical  and 
operational  use  of  existing  unmanned  systems  are  set¬ 
ting  de  facto  rules  of  engagement  and  expectations  do¬ 
mestically  and  internationally.  It  is  unlikely  that  any 
practical  and  authoritative  strategy  and  governance 
will  emerge  that  can  embrace  the  full  scope  of  current 
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autonomous  systems,  let  alone  keep  pace  with  future 
developments.  With  this  in  mind,  the  onus  of  consid¬ 
ering  the  broader  context  and  strategic  implications  is 
best  practiced  by  individual  stakeholders  for  the  near 
future.  Responsible  stakeholders  should  act  coopera¬ 
tively  and  proactively  as  frameworks  of  compliance 
and  best  practices  are  formulated  and  formalized  in 
the  coming  decades. 

Additional  Issues  to  Explore. 

The  scope  of  this  Paper  necessarily  excludes  many 
significant  AWS  issues,  but  two  merit  special  attention 
for  future  studies.  First,  there  needs  to  be  serious  dia¬ 
logue  regarding  how  AWS  principles  apply  to  systems 
that  operate  in  the  domains  of  space  and  cyberspace. 
Cyberspace  systems  are  explicitly  excluded  from  cur¬ 
rent  DoD  policy  on  AWS,  but  these  systems  not  only 
facilitate  the  operation  of  AWS  in  other  domains,  but 
they  may  also  act  as  autonomous  software  agents  in 
operations  limited  to  the  cyberspace  domain.  Second, 
the  use  of  autonomy  and  its  anticipated  proliferation 
need  to  be  included  in  dialogues  regarding  hybrid 
warfare  by  state  and  nonstate  actors. 

CONCLUDING  REMARKS 

The  evolution  of  autonomous  weapon  systems 
accelerated  significantly  after  the  9/11  attacks  on  the 
U.S.  homeland,  and  they  will  likely  continue  to  grow 
in  both  capabilities  and  numbers.  This  growth  of  un¬ 
manned  systems  is  not  limited  to  the  U.S.  military  or 
even  the  United  States;  it  is  an  international  phenom¬ 
ena  that  includes  government  and  commercial  appli¬ 
cations  in  all  domains  — air,  land,  and  sea.  Currently, 
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air-based  systems  dominate  all  aspects  of  AWS  —  bud¬ 
get,  development,  and  operations.  Unmanned  ground 
systems  receive  much  lower  priority  and  are  largely 
relegated  to  niche  applications  vice  being  seriously 
considered  as  part  of  an  integrated  future  land  force. 
Commercial  endeavors  for  unmanned  systems  are  at 
the  forefront  of  many  technologies  and  their  prolifera¬ 
tion  will  likely  outnumber  military  use  in  the  future. 

As  the  U.S.  military  moves  forward  with  the  devel¬ 
opment  of  AWS  doctrine  that  spans  the  full  range  of 
military  operations,  it  may  be  well  served  to  charac¬ 
terize  autonomy  not  as  a  discrete  property  of  a  given 
system,  but  rather  as  a  function  that  varies  in  its  stra¬ 
tegic,  operational,  and  tactical  context  and  mission 
application.  When  can  the  system  be  autonomous? 
When  is  it  planned  in  a  given  mission  to  do  so?  When 
can  it  do  so  by  accident  or  failure?  Efforts  that  address 
such  questions  should  be  tempered  with  the  purpose¬ 
ful  consideration  of  human  judgment  and  control  as 
well  as  legal  and  ethical  standards  that  foster  inter¬ 
national  credibility.  In  many  of  the  challenges  related 
to  autonomous  weapon  systems,  the  United  States  is 
setting  both  overt  and  tacit  precedents  for  the  world  — 
will  we  be  able  to  live  up  to  them  if  they  are  applied 
to  us? 
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and%>  20 Autonomy  %o20to%o20NDIA%o20robotics%>20conference%  20 
6-16-2011-Bobby%20Junker.pdf,  accessed  May  6,  2015,  slide  8. 

47.  DSB,  The  Role  of  Autonomy  in  DoD  Systems,  p.  31. 

48.  Ibid.,  pp.  31-55. 

49.  DoDD  3000.09,  pp.  6, 10. 

50.  Unmanned  Systems  Integrated  Roadmap,  FY2013-2038,  p.  87. 

51.  Ibid.,  p.  69. 

52.  Robotics  Collaborative  Technology  Alliance  (RCTA),  FY 
2012  Annual  Program  Plan,  Adelphi,  MD:  Army  Research  Labora¬ 
tory,  March  2012.  In  addition  to  the  Army  Research  Laboratory, 
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member  of  the  RCTA  are  Boston  Dynamics,  Carnegie  Mellon  Uni¬ 
versity,  Florida  State  University,  General  Dynamics  Robotic  Sys¬ 
tems,  NASA  Jet  Propulsion  Laboratory,  QinetiQ  North  America, 
University  of  Central  Florida,  and  University  of  Pennsylvania. 
The  program  announcement  (February  2,  2009),  which  solicited 
proposals  from  perspective  team  members,  "identified  four  key 
technology  areas  expected  to  be  critical  to  the  development  of  fu¬ 
ture  autonomous  systems,  namely:  Perception,  Intelligence,  Hu¬ 
man-Robot  Interaction  (HRI),  and  Dexterous  Manipulation  and 
Unique  Mobility  (DMUM)."  (p.  1) 

53.  Ibid.,  p.  5.  The  five  cross-cutting  technical  barriers  are: 

Simplistic/Shallow  World  Model.  Existing  autonomous 
systems  fall  into  two  categories:  either  they  have  a  world 
model  that  is  at  only  a  metric  level,  thus  precluding  any 
cognitive  reasoning,  or  they  have  a  model  that  exists  at  only 
a  cognitive  level  without  physical  grounding  in  the  metric 
world.  Neither  approach  is  sufficient  for  our  vision  where 
robots  must  behave  cognitively  while  interacting  in  the 
physical  world. 

Lack  of  Semantic  Understanding.  In  existing  systems,  ob¬ 
jects  in  the  world  are  perceived  primarily  or  only  as  mobility 
regions,  not  as  discrete  objects  of  semantic  and  cognitive  im¬ 
portance.  Thus,  one  cannot  tell  a  robot,  "Go  block  the  back 
door  of  this  building"  and  expect  it  to  do  anything  useful. 
Scripted  and  Brittle  Planning.  Robots  are  almost  always 
tele-operated  or,  at  best,  only  perform  simple  scripted  be¬ 
haviors.  Scripting  all  needed  behaviors  is  not  tractable  and 
does  not  allow  for  learning  new  or  alternative  behaviors. 
Planning  algorithms  in  robots  work  well  only  when  the 
planning  space  is  both  small  and  certain  enough,  but  the  real 
world  is  fraught  with  uncertainty  and  high  dimensionality. 

The  inability  to  reason  in  complex  and  uncertain  environ¬ 
ments  means  that  users  must  intervene  frequently  in  robot 
operations  and  are  trapped  at  a  close  level  of  "supervised 
autonomy." 

No  Shared  Understanding  of  Missions  and  Roles.  Robots 
now  are  opaque  and  distrusted  and  cannot  explain  what 
they  are  doing.  Not  only  do  they  not  know  what  they  are  do¬ 
ing,  but  also  they  do  not  understand  what  their  teammates 
are  doing  or  what  the  expectations  for  roles  and  communi¬ 
cation  are.  Consequently,  current  systems  must  use  tedious 
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OCUs  to  bridge  the  enormous  cognitive  gap  between  hu¬ 
mans  and  robots. 

Missing  or  Shallow  Learning  Capability.  Robots  now 
must  be  explicitly  programmed  to  do  tasks,  so  producing 
the  needed  scope  of  behavior  is  intractable.  Existing  learn¬ 
ing  capability  is  shallow  and  lacks  generalization.  Thus,  we 
cannot  retrain  robots  without  bringing  engineers  to  the  field 
or  sending  the  robots  back  to  the  developer. 


54.  Ibid.,  pp.  2-4. 


55.  Unmanned  Systems  Integrated  Roadmap,  FY2013-2038,  p.  3. 
For  the  chart's  abbreviations:  RDTE  =  Research,  Development, 
Test  and  Evaluation;  Proc  =  Procurement;  and  OM  =  Operations 
and  Maintenance. 

56.  Office  of  the  Under  Secretary  of  Defense  (Comptroller)/ 
CFO,  Program  Acqidsition  Cost  by  Weapon  System,  Fiscal  Year  2016 
Budget  Request,  Washington,  DC:  Department  of  Defense,  Febru¬ 
ary  2015,  pp.  1-1  through  1-5. 

57.  DoD  FY  2016  President's  Budget  Submission :  Army  Justifica¬ 
tion  Book  of  Aircraft  Procurement,  Army,  Washington,  DC:  Depart¬ 
ment  of  the  U.S.  Army,  February  2015,  p.  25.  The  full  description 
for  P-1  Line  Number  A00005,  p.  25,  MQ-1  UAV  is: 

Gray  Eagle  provides  Reconnaissance,  Surveillance,  Target 
Acquisition  (RSTA),  command  and  control,  communica¬ 
tions  relay.  Signals  Intelligence  (SIGINIT),  battle  damage  as¬ 
sessment,  and  manned-unmanned  teaming  capability.  Gray 
Eagle  is  a  dedicated,  assured,  multi-mission  Unmanned  Air¬ 
craft  System  (UAS)  fielded  to  all  ten  Army  Divisions  to  sup¬ 
port  the  commander's  combat  operations.  The  Army  Spe¬ 
cial  Operations  Aviation  Command  (ARSOAC)  Gray  Eagle 
units  and  Aerial  Exploitation  Battalions  (AEB)  Gray  Eagle 
units  are  self-contained  Intelligence,  Surveillance  and  Re¬ 
connaissance  capabilities  teamed  with  organic  Processing, 
Exploitation  and  Dissemination  that  are  a  global  rapidly  de¬ 
ployable  force  and  contribute  to  the  Department  of  Defense 
Global  ISR  mission.  A  Gray  Eagle  unit  consists  of  either  9 
or  12  Aircraft,  Universal  Ground  Control  equipment,  Stan¬ 
dard  Equipment  Package  and  Payloads  to  include:  Electro- 
Optical/  Infrared  Laser  Range  Finder/ Laser  Designator, 
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synthetic  aperture  radar/  ground  moving  target  indicator, 
communication  relay,  and  up  to  4  HELLFIRE  Missiles. 

58.  DoD  FY  2016  President’s  Budget  Submission:  Army  Justifica¬ 
tion  Book  of  Research,  Development,  Test  &  Evaluation,  Army  RDT&E 
-  Volume  II,  Budget  Activity  5a,  Washington,  DC:  Department  of 
the  U.S.  Army,  February  2015,  p.  214.  The  mission  description  for 
the  Tactical  Unmanned  Ground  Vehicle  includes: 

CRS-(I)  is  a  man-packable,  miniature  (<251bs),  highly  mo¬ 
bile,  unmanned  robotic  system  with  advanced  sensors/ mis¬ 
sion  modules  for  dismounted  Soldiers.  The  program  is  the 
result  of  collaboration  between  Director,  Army  Capabilities 
Integration  Center  (DIR  ARCIC),  United  States  Army  Train¬ 
ing  and  Doctrine  Command  (TRADOC)  and  Deputy  Com¬ 
mandant  for  Combat  Development  and  Integration  (DC 
CD&I),  Headquarters  Marine  Corps  (HQMC)  dated  19  Sep 
2012.  Thus  the  CRS-(I)  program  has  been  jointly  developed 
by  the  Army  and  USMC  incorporating  Army  capability 
requirements,  USMC  Engineering  Squad  Robot  (ESR)  and 
USMC  Tactical  Robotic  Controller  (TRC)  capabilities  into 
one  program. 

The  CRS-(I)  capability  contributes  to  the  essential  Joint 
Operational  Concepts  (JOC)  of:  Major  Combat  Operations 
(MCO);  Military  Support  to  Stabilization,  Security,  Transi¬ 
tion,  and  Reconstruction  (SSTR);  Homeland  Support  and 
Civil  Defense  and  Joint  Functional  Concepts  (JFC)  of:  Force 
Application  and  Protection.  The  CRS-(I)  contributes  directly 
to  Situational  Awareness,  Detect,  Protect  and  Neutralize  by 
providing  a  standoff  hazards  interrogation,  detection,  con¬ 
firmation  and  neutralization  capability  employed  to  support 
a  wide  spectrum  of  mobility  missions  for  current  and  future 
forces  by  providing  required  standoff  capability  across  the 
Warfighting  Functions.  This  capability  allows  commanders 
to  make  more  informed  decisions  and  plans,  to  use  their 
forces  more  effectively  and  efficiently  to  produce  desired 
outcomes,  and  to  conduct  focused  operations  for  high-risk 
missions  or  selected  missions  that  best  satisfy  the  require¬ 
ment  without  the  limitations  and  vulnerabilities  of  manned 
systems.  The  CRS-(I)  capability  provides  commanders  the 
ability  to  persistently  monitor  the  operational  environment 
(OE)  while  protecting  and  sustaining  the  force  at  standoff 
distances  from  the  threat.  The  CRS-(I)  complements  the 
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Joint  Integrated  Warfighting  Force  by  providing  standoff  to 
the  Warfighter  during  Major  Combat  Operations,  stability 
operations,  and  homeland  security.  The  CRS-(I)  provides 
Warfighters  the  capability  to  find  and  identify  targets  of  in¬ 
terest  in  the  operational  environment. 

59.  Ibid.,  p.  214,  includes  further  details  on  the  use  of  COTS 
and  other  similar  resources: 

In  support  of  emerging  requirements,  the  Robot  Enhance¬ 
ment  Program  (REP)  uses  a  'buy,  try  and  inform'  methodol¬ 
ogy  to  evaluate  Commercial  Off  The  Shelf  (COTS),  Govern¬ 
ment  Off  The  Shelf  (GOTS)  and  Non-Developmental  Items 
(NDI)  products  that  have  the  potential  to  enhance  Soldier 
combat  effectiveness.  Hardware  quantities  will  be  limited 
to  available  REP  funds.  Evaluation  results  obtained  will 
be  used  to  inform  emerging  requirements  documents  and 
Cost-Benefit  Analyses  to  support  future  Army  decisionmak¬ 
ing  actual  operational  user  feedback. 

60.  Department  of  Defense,  Fiscal  Year  (FY)  2016  President's 
Budget  Submission :  Army  Justification  Book  of  Research,  Development, 
Test  &  Evaluation,  Army  RDT&E-Volume  II,  Budget  Activity  4, 
Washington,  DC:  Department  of  Defense,  February  2015,  p.  324. 

61.  Jon  Bronstein  and  Bob  Mitchell,  "Robotics  Collaborative 
Tecfinology  Alliance  — Foundations  of  Autonomy  for  Ground 
Robotics,"  program  overview  briefing,  Adelphi,  MD:  Army  Re¬ 
search  Laboratory,  April  26,  2012,  slide  2,  available  from  www. 
arl.army.mil/www/pages/392/OvervieioBriefingRoboticsCTA.pdf 
accessed  on  May  6,  2015. 

62.  Unmanned  Systems  Integrated  Roadmap,  FY2013-2038, 
pp.  23-25. 


63.  DSB,  The  Role  of  Autonomy  in  DoD  Systems,  p.  92. 

64.  Ibid.,  p.  19. 

65.  DoDD  3000.09,  p.  12. 

66.  Unmanned  Systems  Integrated  Roadmap,  FY2013-2038, 
p.  18.  The  benefits  of  integrating  unmanned  systems  into  existing 
force  structure  include: 
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Unmanned  systems  open  up  new  avenues  for  pursuing  sys¬ 
tems  that  are  smaller,  lighter,  faster,  and  more  maneuver- 
able  and  that  take  more  risk  than  equivalent  manned  plat¬ 
forms.  In  particular,  the  ability  of  unmanned  assets  to  take 
risks  that  would  not  be  taken  with  manned  assets  opens  up 
new  CONOPS,  such  as  low-cost,  expendable  systems  that 
trade  armor  and  stealth  for  quantity.  In  other  words,  a  fleet 
of  low-cost,  disposable  platforms  could  survive  through  at¬ 
trition  rather  than  through  expensive,  exquisite  capabilities. 

67.  Pernin  et.  al.,  Lessons  from  the  Army's  Future  Combat  Systems 
Program ,  pp.  236-238. 

68.  Ibid.,  p.  43. 

69.  Robotics  Systems  JPO,  Unmanned  Ground  Systems  Road¬ 
map,  pp.  41-43. 

70.  Ibid.,  p.  43. 

71.  Ibid.,  p.  44. 

72.  DSB,  The  Role  of  Autonomy  in  DoD  Systems,  p.  16.  The 
report  provides  further  details  on  the  challenge  of  creating  an 
autonomous  system  that  can  follow  complex  instructions  in  a 
dynamic  environment: 

Terrain  negotiation  and  obstacle  avoidance  are  driven  by 
mechanical  capabilities  coupled  with  pattern  recognition 
and  problem  solving  skills.  Operations  within  the  ROE, 
however,  represent  a  higher  order,  biomimetic  cognitive 
skill  that  must  fall  within  the  commander's  intent.  Going 
forward,  development  efforts  should  aim  to  advance  tech¬ 
nologies  to  better  overcome  these  challenges.  Particularly 
in  the  latter  case,  the  development  of  autonomous  systems 
that  allow  the  operator/  commander  to  delegate  specific 
cognitive  functions,  that  may  or  may  not  change  during  the 
course  of  a  mission  or  engagement,  would  appear  to  be  an 
important  milestone  in  evolution  from  remotely  controlled 
robotics  to  autonomous  systems. 

73.  Robotics  Systems  Joint  Project  Office,  Unmanned  Ground 
Systems  Roadmap,  Washington,  DC:  Department  of  the  Army,  July 
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2011,  p.  41-43.  Appendix  A  (pp.  56-71)  of  this  roadmap  provides 
descriptions  and  specifications  for  the  systems  and  programs 
overseen  by  the  Robotics  Systems  Joint  Program  Office  at  that 
time.  They  were: 

Anti-Personnel  Mine  Clearing  System,  Remote  Control 
(M160) 

PackBot  Family  of  Systems 
Mini-EOD  [explosive  ordnance  disposal] 

TALON  Family  of  Systems 
MARCbot 

Common  Mobility  Platform  (CMP) 

XM1216  Small  Unmanned  Ground  Vehicle  (SUGV) 

XM155  Autonomous  Navigation  System  (ANS) 

74.  Unmanned  Systems  Integrated  Roadmap,  FY2013-2038,  p.  19. 
A  more  detailed  description  of  the  MUM-T  concept  is  included  in 
Appendix  E: 

The  concept  of  MUM-T  is  to  combine  the  inherent  strengths 
of  manned  platforms  with  the  strengths  of  UAS,  with  prod¬ 
uct  synergy  not  seen  in  single  platforms.  MUM-T  combines 
robotics,  sensors,  manned/ unmanned  vehicles,  and  dis¬ 
mounted  soldiers  to  achieve  enhanced  situational  aware¬ 
ness,  greater  lethality,  improved  survivability,  and  sustain¬ 
ment.  Properly  designed,  MUM-T  extends  sensor  coverage 
in  time  and  space  and  provides  additional  capability  to 
acquire  and  engage  targets. 

The  pilot  can  use  the  sensor  on  the  UAS,  just  as  a  sensor 
would  be  used  aboard  an  aircraft,  except  that  the  position  of 
the  UAS  sensor  can  be  up  to  80  km  ahead  from  the  aircraft. 

The  MUM-T  capability  provides  an  unprecedented  standoff 
range  from  threat  weapons  and  acquisition  systems.  MUM 
systems  largely  depend  on  mission,  enemy,  terrain,  troops, 
time,  and  civil  considerations.  The  transfer  of  sensor  data 
between  the  UAS  and  the  manned  system  reduces  risk  to 
both  platforms  and  increases  the  mission  effectiveness  and 
survivability  rates  of  friendly  forces.  Environmental  condi¬ 
tions  affect  the  efficiency  of  MUM-T  employment,  (p.  139) 

75.  Ibid.,  p.  139-141. 

The  MUSIC  exercises  showcase  to  the  soldier  and  Army 
community  the  progress  being  made  in  unmanned  interop- 
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erability  and  emerging  technologies  through  common  in¬ 
terfaces.  Exercises  also  act  as  a  strategic  planning  tool  by 
driving  integration  and  test  of  the  various  platforms  to  a 
common  hardware  and  software  baseline. 

Overview.  The  objective  for  the  MUSIC  I  Exercise  was  to 
showcase  interoperability  progress  and  emerging  technolo¬ 
gies  in  accordance  with  the  2.x  series  of  the  Army  UAS  IOPs. 

The  exercise  took  place  in  Dugway  Proving  Grounds,  Utah, 
at  the  UAS  Rapid  Integration  &  Acceptance  Center  on  16 
September  2011.  Weeks  of  pre-ground  and  -flight  checks 
culminated  into  a  live  two-hour  demonstration  to  a  group 
of  media,  contractors,  and  Army  officials.  The  audience 
witnessed  real-time  video  feeds  from  the  unmanned  and 
manned  payloads,  screen  captures  from  the  GCSs,  video 
feeds  from  within  the  shelter,  and  visual  aids  through  an 
operational  scenario  to  help  demonstrate  the  capabilities 
and  achieve  a  better  understanding  of  how  they  benefit  the 
soldier.  Successful  execution  of  the  MUSIC  exercise  provid¬ 
ed  the  product  office  with  a  wide  range  of  lessons  learned 
across  multiple  areas,  including  system  usability,  reliability, 
integration,  and  configuration  control,  (p.  139) 

76.  Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics,  Report  to  Congress  on  Addressing  Chal¬ 
lenges  for  Unmanned  Aircraft  Systems,  Washington,  DC:  Depart¬ 
ment  of  Defense,  September  2010,  p.  3.  The  UAS  TF  Charter  iden¬ 
tified  goals  for  the  organization  as: 

1.  Coordinate  and  evaluate  DoD  UAS  requirements,  remain¬ 
ing  constantly  conscious  of  technology,  cost,  schedule,  joint¬ 
ness,  and  interoperability  imperatives; 

2.  In  coordination  with  key  UAS  stakeholders,  increase  the 
operational  effectiveness  of  DoD  UAS  by  promoting  the 
development  and  fielding  of  interoperable  systems  and  net¬ 
works; 

3.  Shape  DoD  UAS  acquisition  programs  to  prioritize  joint 
solutions  which  guarantee  interoperability,  efficient  produc¬ 
tion,  lower  unit  costs,  decreased  support  costs,  and  increased 
capability; 

4.  Serve  as  the  DoD's  advocate  for  shaping  the  regulatory 
policies,  procedures,  certification  standards,  and  technology 
development  activities  that  are  critical  to  the  integration  of 
Department  UAS  into  the  national  airspace  system  to  fulfill 
future  operational  and  training  requirements;  and 
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5.  Serve  as  the  Department's  lead  activity  for  the  develop¬ 
ment  and  promulgation  of  the  Unmanned  Systems  Road¬ 
map.  (pp.  3-4) 

77.  Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics,  Report  to  Congress  on  Addressing  Chal¬ 
lenges  for  Unmanned  Aircraft  Systems,  Washington,  DC:  Depart¬ 
ment  of  Defense,  January  2013,  p.  2.  The  goals  for  UAS  TF  inte¬ 
grated  product  teams  are: 

The  [Interoperability]  I-IPT  continues  to  lead  the  DoD  ef¬ 
fort  championing  more  efficient  acquisition  policies  and 
practices,  promoting  Joint  standards  and  architectures,  and 
influencing  common  practices  that  enhance  manned  and 
unmanned  interoperability  across  the  Joint,  Interagency, 
Intergovernmental,  and  Multinational  (JIIM)  domains,  (p.  1) 

The  Frequency  and  Bandwidth  IPT  (FB-IPT)  is  tasked  to  de¬ 
velop  DoD  spectrum  and  bandwidth  guidance  for  improv¬ 
ing  UAS  operational  effectiveness  and  mission  capability 
for  large,  medium,  and  small  platforms  via  new  technical 
capabilities;  operational  TTPs;  and  regulatory  actions.  The 
FB-IPT  represents  DoD  and  upholds  the  U.S.  frequency 
and  bandwidth  policies  and  guidance  associated  with  UAS 
around  the  world,  (p.  12) 

The  goal  of  Logistics  and  Sustainment  IPT  (LS-IPT)  is  de¬ 
velopment  of  affordable  DoD-wide  approaches  for  long¬ 
term  sustainment  of  UAS  capabilities  that  support  systems' 
readiness  objectives.  The  LS-IPT  provides  a  forum  to  discuss 
common  issues  and  opportunities  for  synergies  between 
programs  aimed  at  assisting  UAS  programs  and  the  Mili¬ 
tary  Departments  in  the  development  of  UAS  sustainment 
strategies  by  leveraging  experience  from  existing  programs, 
organizations,  disciplines,  and  processes  to  identify  best 
practices  and  lessons  learned,  (p.  15) 

78.  Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics,  Report  to  Congress  on  Addressing  Chal¬ 
lenges  for  Unmanned  Aircraft  Systems,  Washington,  DC:  Depart¬ 
ment  of  Defense,  December  2013,  p.  1. 

79.  DoDD  3000.09,  pp.  7-8.  Enclosure  3,  paragraph  l.b.  in¬ 
cludes:  b.  Before  fielding,  the  USD(P),  USD(AT&L),  and  CJCS 
shall  ensure: 
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(1)  System  capabilities,  human-machine  interfaces,  doctrine, 
TTPs,  and  training  have  demonstrated  the  capability  to  al¬ 
low  commanders  and  operators  to  exercise  appropriate  lev¬ 
els  of  human  judgment  in  the  use  of  force  and  to  employ 
systems  with  appropriate  care  and  in  accordance  with  the 
law  of  war,  applicable  treaties,  weapon  system  safety  rules, 
and  applicable  ROE. 

(4)  Adequate  training,  TTPs,  and  doctrine  are  available,  pe¬ 
riodically  reviewed,  and  used  by  system  operators  and  com¬ 
manders  to  understand  the  functioning,  capabilities,  and 
limitations  of  the  system's  autonomy  in  realistic  operational 
conditions. 

(5)  System  design  and  human-machine  interfaces  are  read¬ 
ily  understandable  to  trained  operators,  provide  traceable 
feedback  on  system  status,  and  provide  clear  procedures 
for  trained  operators  to  activate  and  deactivate  system 
functions. 

80.  P.  W.  Singer,  Wired  for  War:  Tire  Robotics  Revolution  and 
Conflict  in  the  Twenty-first  Century,  New  York,  NY:  The  Penguin 
Press,  2009,  p.  210. 

81.  Junker,  "ONR 101  &  Navy  Autonomy  Issues  and  Needs," 
slide  10. 

82.  DSB,  The  Role  of  Autonomy  in  DoD  Systems,  p.  45. 

83.  Junker,  "ONR  101  &  Navy  Autonomy  Issues  and  Needs," 
slide  3. 

84.  Brian  P.  Bedell,  "Small  Ground  Robot's  Effectiveness  and 
Acquisition  Strategy,"  Civilian  Research  Project,  Carlisle  Bar¬ 
racks,  PA:  U.S.  Army  War  College,  June  2010,  p.  84. 

85.  DSB,  The  Role  of  Autonomy  in  DoD  Systems,  p.  50.  The  re¬ 
port  provides  more  detail  on  the  concepts  of  cooperation  and 
collaboration: 

Synchronization  is  often,  but  not  universally,  referred  to 
as  cooperation  by  multi-agent  systems  researchers,  with 
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2001-2002,  pp.  57-71.  In  his  article,  Adams  raises  cautions  about 
certain  aspects  of  disengagement  facilitated  by  technology: 

Automated  systems  can  certainly  reduce  the  pressure  of  in¬ 
formation  saturation  and  eliminate  conflicts,  but  at  a  price. 

Essentially,  they  do  so  by  creating  a  series  of  information 

'filters'  that  establish  priorities  and  eliminate  marginal  data, 

reconcile  the  remaining  information  conflicts,  and  present 
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a  consensus  picture  of  the  situation.  All  of  this  is  invisible 
to  the  ultimate  consumer,  out  of  his  or  her  control  and  very 
likely  not  well  understood.  This  means  that  the  commander 
is  receiving  a  picture  of  the  battlefield  that  is  designed  to 
emphasize  certain  things  while  de-emphasizing  others.  Still 
other  factors  are  omitted  entirely,  (p.  62) 

In  sum,  this  approach  results  in  the  development  of  systems 
that  take  the  operator  'out  of  the  loop,'  shifting  the  role  of 
the  human  operator  from  that  of  an  active  controller  to  that 
of  a  supervisor  who  serves  in  a  fail-safe  capacity  in  the  event 
of  system  malfunction.  Unfortunately,  the  role  of  passive 
monitor  seems  to  be  a  task  for  which  humans  are  poorly 
suited,  (p.  64) 
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